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PREFACE 


This  document  contains  the  proceedings  from  the  Defense  Modeling  and  Simula'ion  Office 
(DMSO)  Information/Data  Base  Technology  Working  Group  (I/DBTWG)  meeting  and  related 
Task  Force  meetings  held  at  the  institute  for  Defense  Analysis  (IDA)  during  the  week  of  July 
1 1-15,  1994.  (Note  that  the  name  of  the  1/DBTWG  has  been  changed  from  the  previous  name 
I/DB  Task  Group  (1/DBTG)). 

The  work  described  here  was  performed  for  the  Defense  Modeling  and  Simulation  Office  as  part 
of  its  initiative  to  strengthen  the  use  of  simulation  and  modeling  throughout  DoD.  RAND’s 
participation  in  this  effort  was  performed  for  the  Director,  Defense  Modeling  and  Simulation 
Office  within  the  Applied  Science  and  Technology  Program  of  RAND's  National  Defense 
Research  Institute  (NDRI),  a  federally  funded  research  and  development  center  sponsored  by  the 
Office  of  the  Secretary  of  Defense  and  the  Joint  Staff. 

This  work  should  be  of  interest  to  those  working  in  the  areas  of  interoperability  of  information 
systems,  information  resource  management  (IRM),  data  dictionary  systems,  resource  directories, 
data  modeling  and  use  of  IDEF  tools,  complex  data,  data  verification,  validation,  and 
certification  (VV&C),  data  quality,  and  assessment  of  data  management  technology. 
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SUMMARY 


This  document  contains  the  proceedings  from  the  Defense  Modeling  and  Simulation  Office 
(DMSO)  Information/Data  Base  Technology  Working  Group  (I/DBTWG)  meeting  and  related 
Task  Force  meetings  held  at  the  Institute  for  Defense  Analysis  (IDA)  during  the  week  of  July 
1 1-15, 1994.  (Note  that  the  name  of  the  I/DBTWG  has  been  changed  from  the  previous  name 
I/DB  Task  Group  (I/DBTG)) 

The  DMSO  I/DBTWG  was  formed  in  January  1992  from  the  Information  Technology  and  Data 
Base  Technology  working  groups  who  met  from  August  1991  through  December  1991  to 
perform  technology  assessments  in  support  of  the  DMSO  Master  Plan.  The  original  TWGs  were 
mainly  composed  of  representatives  from  federally  funded  research  and  development  centers 
(FFRDCs).  Two  documents  describing  earlier  I/DBTWG  activities  are:  (1)  “Defense  Modeling 
and  Simulation  Office  Information/Data  Base  (I/DB)  Task  Group  Meetings  Held  February  14- 
18, 1994,  and  Notes  from  the  Previous  Two  I/DB  Meetings,”  RAND  CF-1 14-DMSO,  1994,  and 
(2)  “Database  Technology  Activities  and  Assessment  for  Defense  Modeling  and  Simulation 
Office  (DMSO)  (August  1991 —November  1992),"  RAND  MR-130-ACQ,  1994. 

Because  data  is  critical  to  Modeling  and  Simulation,  the  main  and  continuing  purpose  of  the 
I/DBTWG  is  to  provide  the  M&S  community,  at  low  cost  and  with  efficiency,  timely,  verified 
and  valid  data  to  promote  reuse  and  sharing  of  data,  interoperability  of  models  and  simulations, 
and  improved  credibility  of  modeling  and  simulation  results. 

The  DoD  Corporate  Information  Management  (CIM)  initiative  continues  to  address  many  of  the 
data  related  needs  of  the  M&S  community  but  not  all.  It  is  important  for  the  M&S  community  to 
be  aware  of  the  data  needs  not  being  met  by  CIM  and  unlikely  to  be  met  by  commercial  or  other 
DoD  means.  These  data  needs  should  be  brought  to  the  attention  of  the  Defense  Data 
Administration  Program  Management  Office  (DAFMO)  and  if  DoD  attention  to  them  is  not 
timely  or  inappropriate,  then  they  should  be  addressed  by  the  M&S  community.  It  is  critical  that 
the  I/DBTWG  continue  to  monitor  DoD  CIM  activities  and  help  DMSO  develop  compatible 
M&S  guidelines  and  procedures  whenever  possible. 

The  I/DBTWG  is  currently  co-chaired  by  Dr.  Chien  Huo  from  the  DISA/JIEO  Center  for 
Standards  who  is  working  with  the  DMSO  to  carry  out  their  data  administration  and  standards 
program,  and  by  Ms.  Iris  Kameny  from  RAND  who  led  the  first  Data  Base  Technology  Working 
Group  and  has  been  supporting  DMSO  since  1991  in  their  data  related  activities.  Dr.  Huo  and 
Ms.  Kameny  are  working  with  CDR  Gary  Misch  (DMSO)  and  with  LTC  (P)  Jerry  Wiedewitsch. 
the  Deputy  Director  of  DMSO. 

The  I/DBTWG  has  grown  from  around  a  dozen  members  at  its  inception  to  over  100  members 
today.  It  consists  of  people  from  the  Services,  Joint  Staff,  DoD  agencies.  Intelligence 
Community,  ARPA,  NIST,  NASA,  OSD,  FFRDCs,  and  contractors  working  on  government 
M&S  programs.  The  I/DBTWG  currently  meets  twice  a  year  with  the  next  meeting  scheduled 
for  February  6-10,  1995  at  the  Institute  for  Defense  Analysis  in  Alexandria,  Virginia.  The 
I/DBTWG  has  created  three  Task  Forces  which  have  Subgroups  each  of  which  has  co-chairs  who 
are  predominantly  from  the  Services  and  the  Joint  Staff.  The  1/DBTWG  Task  Forces  and  their 
Subgroups  meet  more  frequently  as  needed. 

Because  of  its  size,  the  I/DBTWG  has  become  more  of  an  information  exchange  forum  for  the 
data  suppliers  to  the  M&S  community  than  an  action  body.  Members  make  requests  for 
information  mainly  about  data  standards,  repositories,  directories,  data  quality,  complex  data,  etc. 
and  the  meeting  agenda  is  developed  according  to  the  expressed  needs.  In  addition,  I/DBTWG 
members  and  others  are  invited  to  brief  about  their  M&S  projects,  database  environments,  and 
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centers  to  support  M&S,  and  non-M&S  oriented  databases  and  systems  used  by  the  M&S 
community.  This  exchange  has  been  very  helpful  in  getting  different  organizations  to  know  each 
other  and  work  together  toward  exchanging  and  reusing  databases  rather  than  developing 
redundant  databases. 

Accomplishments  of  the  I/DB  include: 

—  Developing  the  M&S  Information  System  at  DTIC  and  the  I/DBTWG  portion  on  an 
internet  gopher  server  at  RAND. 

—  Development  of  initial  data  models  and  standards  for  a  Database  Directory  and  a  Model 
and  Simulation  Directory  (each  can  be  used  as  a  “standard”  core  by  different 
organizations  enabling  sharing  of  directory  information  across  the  M&S  community). 

—  Carrying  out  an  initial  pilot  study  of  modeling  complex  derived  data  using  the  Army 
TRAC  weapon  performance  data  (e.g.,  probability  hit,  kill)  and  sharing  the  lessons 
learned  with  the  community. 

—  Development  of  a  methodology  to  build  subject  area  information  data  models  through 
reverse  engineering,  and  training  organizations  in  carrying  out  these  activities  utilizing 
IDEF  modeling  techniques  (the  Joint  Data  Base  Elements  project). 

—  Supporting  DMSO  in  becoming  the  delegated  Functional  Data  Administrator  (FDAd)  for 
the  M&S  functional  area. 

—  Currently  developing  a  Data  Administration  Strategic  Plan  (DASP). 

—  Being  instrumental  in  getting  CIM  to  address  complex  data  and  derived  data  in  their  new 
Defense  Integrated  Repository  System  data  model. 

—  Defined  complex  data  as  “data  which  is  difficult  or  awkward  to  model  using  commonly 
existing  techniques  (i.e.,  IDEF1X  and  ether  kinds  of  Entity-Relationsnip  modeling)  and 
defined  non-exhaustive  categories  of  data  meeting  the  definition  of  being  “complex”  as: 
inheritance,  composition,  derivations,  mappings,  and  artifacts  of  legacy  systems. 

—  Defined  data  verification,  validation,  and  certification  for  data  producers  and  users  . 

To  expedite  work  in  data  related  support  for  M&S,  the  I/DBTWG  has  three  Task  Forces  in  the 
areas  of  Data  Verification,  Validation,  and  Certification  (VV&C),  Data  Standards,  and  Complex 
Data.  Each  of  these  groups  met  during  the  week  <  f  July  11-15, 1994. 

Specific  tasks  being  addressed  by  the  Data  VV&C  Task  Force  include: 

—  Develop  guidelines  for  data  VV&C  including  definition  of  a  data  quality  profile  to 
describe  the  condition  of  a  dataset  or  database  (e.g.,  completeness,  accuracy,  resolution, 
audit  trail,  derivation,  and  the  V&V  tests  applied  to  the  data)  and  requiring  that  the  profile 
be  part  of  the  certification  process. 

—  Addressing  Authoritative  Data  Sources:  developing  taxonomy  of  data  areas,  identifying 
authorities  for  those  areas,  developing  a  directory  of  the  authorities,  and  a  guideline  to 
responsibilities  of  an  Authoritative  Data  Source.  Defining  the  roles  of  M&S  data  centers 
that  receive  data  from  authoritative  sources  and  prepare  it  for  input  to  models. 
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—  Working  with  the  Distributed  Interactive  Simulation  (DIS)  M&S  Verification,  Validation 
and  Accreditation  (VV&A)  WG  and  the  DMSO  VV&A  TWG  to  integrate  the  data 
VV&C  process  into  DIS  and  non-DIS  VV&A  processes. 

Specific  tasks  being  addressed  by  the  Data  Standards  Task  Force  include: 

—  Developing  the  requirements  for  an  M&S  Information  Resources  Repository  system  that 
can  meet  M&S  repository  needs  for  managing  metadata,  instance  data,  and  models  and 
simulations. 

—  Writing  a  “white  paper”  to  present  to  the  DIS  Workshop  on  “DIS  Needs  for  DoD  Data 
Standards." 

Specific  tasks  being  addressed  by  the  Complex  Data  Task  Force  include: 

—  Exploring  the  use  of  advanced  modeling  tools  for  modeling  complex  data  in  a  more  user 
friendly  way. 

—  Performing  pilot  studies  on  complex  data,  in  particular  on  weapons  performance  data  for 
the  Navy  and  Air  Force  to  complement  the  previous  pilot  study  of  Army  weapons 
performance  data. 

—  Defining  and  developing  a  taxonomy  or  index  (e.g.,  keywords,  phrases)  to  support  access 
to  models,  simulations,  and  databases  for  browsing  and  reuse. 

Tasks  to  be  performed  by  the  future  Data  Security  Requirements  Task  Force 

—  Address  the  M&S  data  security  requirements  including:  the  need  to  access  and  acquire 
data  from  different  classification  levels,  data  aggregation,  multi-level  metadata,  etc. 
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1.  INTRODUCTION 


PURPOSE 

The  purpose  of  this  document  is  to  provide  the  proceedings  of  the  July  11-15  Defense  Modeling 
and  Simulation  Office  (DMSO)  Information/Data  Base  Technology  Working  Group  (I/DBTWG) 
meeting  to  members,  to  provide  ^formation  to  people  who  wish  to  participate  in  the  I/DBTWG, 
and  those  with  an  interest  in  data  activities  related  to  modeling  and  simulation. 

BACKGROUND 

In  1991  the  Deputy  Secretary  of  Defense  instituted  a  major  new  initiative  to  strengthen  the 
application  of  modeling  and  simulation  (M&S)  in  the  DoD.  Its  purpose  is  to  promote  the 
effective  and  efficient  use  of  M&S  in  joint  education,  training  and  military  operations,  research 
and  development,  test  and  evaluation,  analysis,  and  production  and  logistics  by:  (1)  establishing 
OSD  cognizance  and  facilitating  coordination  among  DoD  M&S  activities;  (2)  promoting  the  use 
of  interoperability  standards  and  protocols  where  appropriate;  and  (3)  stimulating  joint  use,  high 
return  on  M&S  investment.  Achievement  of  these  goals  requires  the  development  and 
implementation  of  a  DoD  M&S  policy,  establishment  of  a  DoD-wide  management  structure  to 
coordinate  joint  M&S  activities  and  requirements,  and  the  formulation  and  implementation  of  a 
long  range  M&S  joint  investment  strategy. 

A  DoD  Executive  Council  for  Modeling  and  Simulation  (EXCIMS)  consisting  of  DoD 
Component  representatives  was  established  as  a  board  to  advise  the  USD(A&T)  on  M&S  policy, 
initiatives,  M&S  standards,  and  investments  for  improving  current  M&S  capability  and 
promising  M&S  advanced  technologies.  The  Defense  Modeling  and  Simulation  Office  (DMSO) 
was  established  to  serve  as  an  executive  secretariat  for  the  EXCIMS  and  to  provide  a  full-time 
focal  point  for  information  concerning  DoD  M&S  activities.  The  DMSO  promulgates 
USD(A&T))  directed  M&S  policy,  initiatives,  and  guidance  to  promote  cooperation  among  DoD 
Components  to  maximize  M&S  efficiency  and  effectiveness. 

To  carry  out  its  functions  and  develop  a  master  plan,  the  DMSO  enlisted  the  help  of  several 
Federally  Funded  Research  and  Development  Centers  (FFRDCs).  A  number  of  functional  and 
technology  working  groups  were  established  to  determine  the  M&S  needs  and  to  evaluate  the 
state-of-the-art  with  respect  to  those  needs.  The  Functional  Working  Groups  are:  Education, 
Training  and  Military  Operations  (ETMO);  Research  and  Development;  Test  and  Evaluation; 
Analysis;  and  Production  and  Logistics.  The  current  Technology  Working  Groups  are: 
Architecture;  Information  and  Databases  (I/DB);  Verification,  Validation  and  Accreditation; 
Networking;  Environmental  Representation;  Computer  Generated  Forces;  Human  System 
Interfaces;  and  Interoperability  with  C3I  Systems.  The  current  I/DB  grew  out  of  two  original 
TWGs:  the  Information  TWG  and  the  Data  Base  TWG. 

During  initial  startup  activities,  the  Information  Technology  Working  Group  (ITWG)  began  to 
develop  plans  and  design  of  a  DMSO  Information  System  to  facilitate  coordination  among  DoD 
M&S  activities.  The  Database  Technology  Working  Group  (DBTWG)  identified  three  efforts 
found  critical  to  M&S  needs:  need  for  directories,  dictionaries,  encyclopedias,  and  repositories 
to  support  timely  and  cost  effective  access  to,  acquisition  of,  and  validation  of  external  and 
derived  databases;  interoperability,  data  integrity  and  consistency  across  distributed  databases 
and  simulations;  and  M&S  community  objective  assessment  of  data  management  products  such 
as  relational  DBMSs.  COL  Jim  Shiflett  of  DMSO,  asked  that  the  two  TWGs  be  joined  into  the 
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I/DB  Task  Group  which  was  done.  Recently,  upon  DMSO  resurrection  of  the  TWGs,  the  name 
of  the  I/DB  Task  Group  was  changed  to  the  I/DBTWG. 

Two  documents  describing  previous  I/DBTWG  activities  are:  ( 1)  “Defense  Modeling  and 
Simulation  Office  Information/Data  Base  (I/DB)  Task  Group  Meetings  Held  February  14-18, 
1994,  and  Notes  from  the  Previous  Two  I/DB  Meetings,”  RAND  CF-1 14-DMSO,  1994,  and  (2) 
“Database  Technology  Activities  and  Assessment  for  Defense  Modeling  and  Simulation  Office 
(DMSO)  (August  1991— November  1992),”  RAND  MR-130-ACQ,  1994. 

The  I/DBTWG  Group  is  currently  co-chaired  by  Dr.  Chien  Huo  from  the  DISA/JIEO  Center  for 
Standards  who  is  working  with  the  DMSO  to  carry  out  their  data  administration  and  standards 
program,  and  by  Ms.  Iris  Kameny  from  RAND  who  led  the  first  Data  Base  Technology 
Working  Group  and  has  been  supporting  DMSO  since  1991  in  their  data  related  activities.  Dr. 
Huo  and  Ms.  Kameny  work  with  CDR  Gary  Misch  (DMSO)  and  with  LTC  (P)  Jerry 
Wiedewitsch,  the  Deputy  Director  of  DMSO. 

The  I/DBTWG  has  grown  from  around  a  dozen  members  at  its  inception  to  over  100  members 
today.  It  consists  of  people  from  the  Services,  Joint  Staff,  DoD  agencies.  Intelligence 
Community,  ARPA,  NIST,  NASA,  OSD,  FFRDCs,  and  contractors  working  on  government 
M&S  programs.  The  I/DBTWG  currently  meets  twice  a  year  with  the  next  meeting  scheduled 
February  6-10,  1995  at  the  Institute  for  Defense  Analysis  in  Alexandria,  Virginia.  The 
I/DBTWG  has  created  three  Task  Forces  which  have  Subgroups  each  of  which  has  co-chairs  who 
are  predominantly  from  the  Services  and  the  Joint  Staff.  The  I/DBTWG  Task  Forces  and  their 
Subgroups  meet  more  frequently  as  needed. 

Because  of  its  size,  the  I/DBTWG  Task  Group  has  become  more  of  an  information  exchange 
forum  for  the  data  suppliers  to  the  M&S  community  than  an  action  body.  Members  make 
requests  for  information  mainly  about  data  standards,  repositories,  directories,  data  quality, 
complex  data,  etc.  and  the  meeting  agenda  is  developed  according  to  the  expressed  needs.  In 
addition,  I/DBTWG  members  and  others  are  invited  to  brief  about  their  M&S  projects,  database 
environments  and  centers  to  support  M&S,  and  non-M&S  oriented  databases  and  systems  used 
by  the  M&S  community.  This  exchange  has  been  very  helpful  in  getting  different  organizations 
to  know  each  other  and  work  together  toward  exchanging  and  reusing  databases  rather  than 
developing  redundant  databases. 

OBJECTIVES  OF  THE  I/DBTWG 

Because  data  is  critical  to  Modeling  and  Simulation,  the  main  and  continuing  objective  of  the 
I/DBTWG  is  to  provide  the  M&S  community,  at  low  cost  and  with  efficiency,  timely,  verified 
and  valid  data  to  promote  reuse  and  sharing  of  data,  interoperability  of  models  and  simulations, 
and  improved  credibility  of  modeling  and  simulation  results.  To  accomplish  this  goal  requires 
data  administration  policies,  procedures,  standards,  and  supporting  tools  compatible  with  those 
of  CIM  and  the  Services.  It  also  requires  access  to  information  throughout  the  M&S  community 
about  what  is  happening  as  well  as  information  about  the  existence  and  availability  of  models 
and  simulations  and  the  data  they  need.  Of  critical  concern  to  the  community  is  the  quality  of  the 
models  and  simulations  as  well  as  the  data  they  use  and  generate. 

Current  Status  in  Meeting  Objectives 

Data  administration  objectives  are  being  addressed  through  the  delegation  cf  M&S  functional 
area  data  administration  responsibilities  to  DMSO.  DMSO  is  now  the  Functional  Data 
Administrator  (FDAd)  for  M&S  and  is  developing  its  first  Data  Administration  Strategic 
Plan  (DASP). 
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Support  for  Data  Standards.  The  Joint  Data  Element  Interoperability  (JDBE)  project  sponsored 
by  DMSO  has  developed  a  methodology  (documented  in  a  Military  Handbook)  to  build  subject 
area  information  models  through  reverse  engineering  of  existing  databases  using  IDEF1X  tools. 
This  is  being  extended  to  support  the  development  of  data  standards.  The  JDBE  project  is 
available  to  M&S  data  projects  for  IDEF  training  and  help  in  developing  their  data  models. 

Directories.  One  of  the  original  M&S  community  requests  (from  all  of  the  functional  working 
groups)  was  for  directories  to  M&S  databases  and  models  and  simulations.  This  is  being 
addressed.  Data  models  for  both  directories  have  been  developed,  have  undergone  community 
consensus  and  are  being  implemented.  These  directories,  various  M&S  data  centers  and  M&S 
program  reuse  libraries  need  a  taxonomy  or  index  (e.g.,  key  words  or  phrases)  to  enable  access  to 
the  stored  objects  and  information  in  a  user  friendly  way  for  browsing  and  reuse.  The  Complex 
Data  Task  Force  has  a  Taxonomy  Subgroup  that  has  developed  initial  taxonomies  for  both 
directories. 

M&S  Information  System.  The  M&S  Information  System  was  developed  to  meet  the  M&S 
community’s  needs  for  access  to  information  and  it  has  become  operational  over  the  past  year. 
An  I/DBTWG  portion  of  the  system  is  maintained  on  an  internet  gopher  server  at  RAND. 

The  Complex  Data  Task  Force.  The  I/DBTWG  recognized  the  lack  of  attention  in  the  CIM 
community  to  data  standards  for  scientific  and  technical  data  and  formed  the  Complex  Data  Task 
Force  to  address  these  needs.  Much  M&S  data  is  not  atomic  single  concept  data  addressed  by 
the  CIM  data  standardization  process  (in  accord  with  DoD  8320.1-M-l)  but  is  complexly  derived 
(e.g.,  probability  hit,  kill),  or  structurally  complex  (e.g.,  a  road  network,  an  object-oriented 
engineering  view  of  a  weapon  system),  or  multimedia  data  (e.g.,  images,  graphics,  voice),  or 
conceptually  complex  (e.g.,  rules,  operation  orders)  data.  A  Complex  Data  Task  Force  task  has 
been  to  define  complex  data  and  to  categorize  it. 

Complex  data  has  been  defined  as  “data  that  is  difficult  or  awkward  to  model  using  commonly 
existing  techniques  (i.e.,  IDEF IX  and  other  kinds  of  Entity-Relationship  modeling).  A  non- 
cxhaustive  set  of  categories  for  complex  data  has  also  been  defined  to  include:  data  represented 
in  object  inheritance  relationships  such  as  multiple  inheritance,  multiple  roots,  and  poiy- 
instantiated  data;  data  represented  in  composition  relationships  such  as  “part-of ’,  complexly 
derived  data  such  as  probability  hit/kill  (as  opposed  to  simply  derived  data  such  as  “age”);  data 
requiring  inter-model  and  intra-model  mappings;  and  composed  data  that  are  artifacts  of  legacy 
systems  (e.g.,  basic  encyclopedia  number). 

This  TF  has  also  been  working  with  the  CIM  Defense  Information  Repository  System  (DIRS) 
project  to  include  complex  and  derived  data  in  its  data  model.  Current  complex  data  tasks 
include  finding  and  trying  out  new  data  modeling  techniques  that  will  be  more  user  friendly  and 
to  perform  additional  pilot  studies  using  these  modeling  techniques  (preferably  for  Air  Force  and 
Navy  weapons  performance  data).  In  addition,  its  Taxonomy  Subgroup  continues  to  address 
taxonomy  issues. 

Data  Standards  Task  Force;  This  Task  Force  has  two  Subgroups:  the  Repository  Subgroup  and 
the  DIS  Standards  Subgroup.  The  Repository  Subgroup  is  developing  the  requirements  for  an 
M&S  Information  Resources  Repository  (IRR)  system  that  can  meet  M&S  repository'  needs  for 
managing  metadata,  instance  data,  and  models  and  simulations.  It  is  to  be  based  on  a  Technical 
Architecture  Framework  for  Information  Management/Technical  Reference  Model 
(TAFIM/TRM)  architecture  with  minimal  standards  and  conventions  and  a  common  tool  base. 
The  prototype  IRR  will  support  the  M&S  FDAd  functions  at  DMSO.  The  long  term  vision  is  of 
a  distributed  confederation  of  IRRs  that  can  seamlessly  exchange  information  resources. 
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The  DIS  Data  Standards  Subgroup  will  be  presenting  a  paper  at  the  September  DIS  Workshop  on 
'‘DIS  Needs  for  DoD  Data  Standards”  and  will  work  with  DIS  to  form  a  working  group  to 
address  DIS  data  standards. 

Data  Verification,  Validation  &  Certification  Task  Force.  The  VV&C  TF  has  formed  two 
Subgroups:  the  VV&C  Guidelines  and  Quality  Profile  Subgroup  and  the  Authoritative  Data 
Sources  Subgroup.  The  Guidelines  Subgroup  is  developing  guidelines  for  the  VV&C  of  data  and 
is  working  closely  with  the  DMSO  VV&A  TWG  and  the  DIS  ,  7&A  Working  Group  to 
integrate  the  data  VV&C  process  into  the  DIS  and  non-DIS  VV&A  processes.  It  is  developing 
guidelines  for  data  VV&C  including  definition  of  a  data  quality  profile  to  describe  the  condition 
of  a  dataset  or  database  (e.g.,  completeness,  accuracy,  resolution,  audit  trail,  derivation,  and  the 
V&V  tests  applied  to  the  data)  and  requiring  the  profile  be  part  of  the  certification  process. 

CtM  has  recently  become  interested  in  promoting  data  quality  within  the  DoD.  The  main 
difference  between  their  data  quality  program  and  the  development  of  the  M&S  quality  profile 
effort  appears  to  be  that  they  are  engaged  in  establishing  data  quality  standards  within  DoD  while 
this  Subgroup  is  trying  to  develop  a  way  to  describe  the  quality  of  a  database  independent  of  a 
quality  standard.  Some  M&S  databases  (e.g.,  intelligence  force  assessments,  futures)  are  by  their 
nature  incomplete,  of  variable  probability  of  belief,  etc.  This  is  the  type  of  information  (as  well 
as  other  kinds  of  data)  that  will  be  captured  in  the  profile. 

The  Authoritative  Data  Sources  Subgroup  is  concerned  with  the  ability  of  M&S  users  to  find  and 
acquire  the  instance  data  they  need  for  their  M&S  and  for  that  data  to  be  VV&Ced,  configuration 
managed,  etc.  This  Subgroup  is  developing  a  taxonomy  of  data  areas,  identifying  authorities  for 
those  areas,  developing  a  directory  of  the  authorities,  and  a  guideline  to  responsibilities  of  an 
Authoritative  Data  Source.  They  are  also  defining  the  roles  of  M&S  data  centers  that  receive 
data  from  authoritative  sources  and  prepare  it  for  input  to  models  and  are  concerned  with  release 
authority  issues. 

Data  Security  Requirements.  An  additional  need  that  will  probably  be  addressed  in  the  near 
future  by  creating  another  task  force  is  to  define  the  data  security  requirements  for  M&S  data  to 
include  the  need  to  access  and  acquire  data  from  different  classification  levels,  data  aggregation, 
multi-level  metadata,  etc. 

ORGANIZATION  AND  STRUCTURE  OF  THIS  DOCUMENT 

This  document  contains  the  proceedings  from  the  Defense  Modeling  and  Simulation  Office 
(DMSO)  Information/Data  Base  Technology  Working  Group  (1/DBTWG)  meeting  and  related 
Task  Force  meetings  held  at  the  Institute  for  Defense  Analysis  (IDA)  during  the  week  of  July 
11-15,1994. 

Section  1  contains  the  introduction. 

Section  2  contains  the  highlights  of  the  I/DBTWG  meetings  during  the  week  of 
July  11-15, 1994. 

Section  3  contains  notes  for  the  main  I/DBTWG  meeting  held  on  July  1 1-12, 1994 
which  includes  an  update  on  DMSO  happenings;  reports  from  I/DBTWG  task  forces 
and  subgroups;  updates  on  data  administration,  standardization  and  modeling  activities; 
reports  about  other  organizations;  reports  from  Service  M&S  organizations  with  respect 
to  data  related  activities;  reports  from  Functional  Working  Groups;  and  reoons  from 
M&S  data  related  projects. 
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Section  4  contains  notes  on  the  repository  discussion  held  on  Monday,  Ju’y  1 1  at  1700 
at  DMSO. 

Section  5  contains  the  notes  for  the  Complex  Data  Task  Force  meeting  held  on 
Wednesday,  July  13,  1994. 

Section  6  contains  the  notes  for  the  Data  Standards  Task  Force  Meeting  held  on  Thursday 
morning,  July  14, 1994. 

Section  7  contains  the  notes  for  the  Data  VV&C  Guidelines  Subgroup  held  on  Thursday 
afternoon,  July  14, 1994. 

Section  8  contains  the  notes  for  the  Data  VV&C  Task  Force  meeting  held  on  Friday,  July 
15,  1994. 

The  Appendices  contain  the  briefing  charts  for  ail  the  meetings  and  a  list  of  acronyms. 


! 
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2.  I/DBTWG  MEETING  HIGHLIGHTS 


The  8th  I/DBTWG  Conference  was  a  success.  Dr.  Chien  Huo  of  DMSO,  Ms.  Iris  Kameny  of 
RAND  and  Services’  representatives  co-cbaired  the  I/DBTWG  plenary  session  and  its  three  task 
forces  (TF)  meetings:  Complex  Data.  Data  Validation,  Verification  and  Certification  (VV&C) 
and  Data  Standards.  Over  60  people  attended,  representing  the  Services,  Joint  Staff,  OSD, 

DMA,  DIA,  JIEO/CIM,  CINCs,  NIST,  NASA,  industry  and  academia.  The  conference’s  main 
theme  was  data  standardization  and  management.  In  addition  to  the  speakers  from  DoD, 
I/DBTWG  also  invited  experts  from  the  NIST,  NASA,  academia,  and  non-govemment  standard 
bodies  such  as  ISO,  IEEE.  X3H2,  X3H4.  There  was  a  positive  interchange  of  information 
throughout  the  conference  agenda. 

Results  from  the  Task  Force  Meetings 

The  I/DBTWG  received  a  list  of  priorities  and  issues  from  each  of  the  task  forces  (TF)  at  the 
conclusion  of  the  meetings.  These  issues  will  be  regarded  as  a  general  guidance  for  the  task 
forces  and  I/DBTWG  to  press  on.  The  top  priority  issues  are  summarized  below: 

a.  Data  Standards  Task  Force: 

—  Develop  procedures  and  guidance  for  the  M&S  data  element  developers  submitting 
proposal  packages  to  DoD  for  data  standardization 

—  Develop  M&S  repository  requirements  to  support  M&S  data  administration  program 
in  accordance  with  the  DDRS/DIRS 

—  Provide  interoperability  across  M&S  data  repositories  ( i.e.,  CENTCOM’s 

Conventional  Force  Database  (CFDB),  J8’s  Operational  Analysis  and  Simulation 
Interface  System  (OASIS),  Automated  Repository  for  M&S  System  (ARMS),  etc.) 

—  Provide  electronic  exchange  of  metadata  and  instance  data  (standard  file  formats, 
exchange  mechanism) 

—  Perform  pilot  studies  of  reverse  engineering  on  CFDB,  data  modeling  on  CFDB  while 
assessing  C2  Core  Data  Model,  forward  engineering  on  Universal  Threat  Simulation 
System  (UTSS),  and  modeling  complex  data  such  as  Pk,  Ph  with  Navy’s  and/or  Air 
Force’s  data 

b.  Data  Verification,  Validation  and  Certification  Task  Force 

—  Incorporate  VV&C  process  into  DIS  VV&A  process  model  and  quick  planner 

—  Develop  IDEFO  process  models  for  user  data  VV&C  for  Non-DIS  applications  (i.e., 
Army,  Navy,  Air  Force,  JS,  OSD) 

—  Finalize  taxonomy  for  authoritative  data  sources 

—  Identify  the  responsibilities  for  data  centers,  authoritative  sources  and  users 

—  Define  how  classified  data  centers  can  exchange  data  with  other  centers  and  release 
data  to  users.  Address  data  aggregation  and  release  authority  issues 

—  Develop  VV&C  Guideline  to  include  database  quality  profile 
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c.  Complex  Data  Task  Force 

—  Perform  pilot  studies  of  complex  data  such  as  Pk.  Ph  with  Navy's  or  Air  Force's  data 

d.  Form  a  new  Task  Force  to  address  data  security  and  MLS  requirements 
Suggested  Briefings  for  Next  1/DBTWG 

(1)  DoD  megacenter  client/server  architecture:  Bill  Tuffy 

(2)  DDRS  update 

(3)  Global  Command  and  Control  System  (GCCS) 

(4)  DoD  data  security  and  quality:  Jerry  Cooper  703-636-6900 
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3.  I/DBTWG  MEETING  NOTES 


3.1  AGENDA 

0800-0830 
0830-0900 

0900-0915 

0915-0930 

0930-0945 

0945-1000 

1000-1030 
1030-1100 


MONDAY,  JULY  li,  1994 
UPDATE  ON  DMSO  HAPPENINGS 

Welcome  and  DMSO  Update:  LTC  (P)  Jerry  Wiedewitsch 

M&S  Data  Administration  Update:  Dr.  Chien  Huo 

REPORTS  FROM  1/DB  TASK  FORCES  AND  SUBGROUPS 

Report  from  M&S  Data  VV&C  Task  Force  Subgroup  on  VV&C 
Guidance  including  Data  Quality  Profile:  Mr.  Bob  Hartling 

Report  from  M&S  Data  VV&C  Task  Force  Subgroup  on  Authoritative 
Data  Sources:  Mr.  Bill  Dunn 

Report  from  M&S  Complex  Data  Task  Force  Subgroup  on  Categorization: 

Mr.  Len  Seligman 

Report  from  M&S  Data  Standards  Task  Force  and  also  on  taxonomies: 

Ms.  Iris  Kameny 

Break 

Report  on  M&S  Directory  and  Databases  Directory  Progress:  Dr.  Mike  Frame 


1100-1130  Report  on  MORS  Simulation  Data  and  Its  Management  (SIMDATAM)  Senior 
Advisory  Group  (SAG):  Mr.  Howard  Haeker 

DATA  ADMINISTRATION,  STANDARDIZATION  AND  MODELING  ACTIVITIES 

1130-1200  Update  on  ASD(C3I)  Data  Standardization  Policies  and  Procedures: 

Mr.  Bob  Molter 

1200-1300  Lunch 

1300-1330  Status  of  Repository  Standards:  Mr.  Bruce  Rosen 
1330-1400  Report  from  Intelligence  FDAd:  Mr.  George  Endicott 
1400-1430  Report  from  Acquisition  and  Technology  FDAd:  Mr.  Gary  Hurd 
1430-1500  Report  from  C2  FDAd:  Mr.  Stan  Plummer 
1500-1530  Break 

1530-1600  JIEO/CIM  Plan  for  M&S  Support:  Ms.  Carla  Von  Bemewitz 

1600-1630  Report  on  ISO  Data  Representation:  Mrs.  Melody  Rood 

1630-1700  Discussion  of  Initiatives  on  External  Data  Standards  and  Migration  Planning: 
Mr  Phil  Cykana 


TIJFSDAY  Jill  Y  12  1994 
REPORTS  ABOUT  OTHER  ORGANIZATIONS 


0800  -0830  Report  on  NASA  Conference  on  Catalog  Interoperability/NASA  Science 
Internet  (CI/NSI):  Ms.  Patricia  Liggett 

0830-0900  Report  on  BMDO  Data  Management:  Mr.  Allen  Hess 

0900-0930  Overview  of  C2  Core  Data  Model:  Dr.  Robert  Walker 

0930-1000  Report  on  IEEE  IDEF IX  Working  Group:  Mr.  Peter  Valentine 

1000-1030  Break 

1030-1100  Report  on  Army  Data  Standards  Organization  Experience  with  IDEF  IX  and 
Data  Standards:  Mr.  Jim  Glymph 

REPORTS  FROM  SERVICE  M&S  ORGANIZATIONS 
WITH  RESPECT  TO  DATA  RELATED  ACTIVITIES 

1 100-1 120  Update  from  Army  Modeling  and  Simulation  Management  Office  (AMSMO): 
Mrs.  Lana  McGlynn 

1120-1 140  Update  from  Air  Force  XOMT:  Lt  Col  Cheryl  Balombini 
1 140-1200  Update  from  Navy  M&S  Office:  Mr.  Dean  Free  and  LCDR  George  Flax 

1200-1300  Lunch 

REPORTS  FROM  FUNCTIONAL  WGS 

1300-1330  DoD  M&S  Master  Plan  Process:  CDR  Mike  Lilienthal 

1330-1400  Report  from  Analysis  Functional  Working  Group:  Mr.  Jim  Heusrrann  for 
Dr.  Pat  Sanders 

1400-1430  Report  from  Production  and  Logistics  Functional  Working  Group: 

Mr.  Fred  Myers 

REPORTS  FROM  M&S  DATA  RELATED  PROJECTS 

1430-1500  Report  on  Environmental  Effects  in  Distributed  Interactive  Simulation  (E2DIS) 
Project:  Dr.  Harry  heckathom 

1500-1530  Break 

1530-1600  Report  on  Master  Environmental  Library:  Dr.  John  Harding 

1600-1630  Report  on  Naval  Battle  Force  Tactical  Trainer  (BFTT):  Mr.  James  Hammond 

1630-1700  Wrapup:  Dr.  Chien  Huo  and  Ms.  Iris  Kamenv 


I/DBTWG  GENERAL  MEETINGS 
MONDAY,  11  JULY  1994 
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3.2  JULY  11  ATTENDEE  LIST  (Cont’d.) 
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3.3  UPDATE  ON  DMSO  HAPPENINGS 

LTC  (P)  Jerry  Wiedewitsch,  Deputy  Director  of  DMSO:  Welcome  and  DMSO  Update 

LTC  (P)  Wiedewitsch  told  the  l/DBTWG  that  the  new  DMSO  direction  is  technically  oriented 
toward  near-term  payoff.  It  is  highly  concerned  with  interoperability  issues.  Examples  of 
synthetic  environment  use  include  American  manufacturing,  operations  other  than  war  such  as 
disaster  relief,  weekend  training,  major  forces  exercises,  and  training  and  wargames.  The 
Defense  Simulation  Internet  (DSI)  is  viewed  as  a  subset  of  the  global  grid/national 
communications  highway.  The  information  highway  will  be  able  to  be  accessed  through  the 
DSI.  There  will  be  funds  for  pilot  projects  especially  in  the  integration  of  live,  virtual  and 
constructive  simulations,  and  integration  of  M&S  into  C4I  systems. 

One  initiative  that  will  be  demonstrated  in  the  first  week  of  November  is  Atlantic  Resolve  (what 
used  to  be  REFORGER)  and  the  Synthetic  Theater  of  War  -  Europe  (STOW-E).  This  will  be  an 
exercise  in  confederated  constructive  simulations  and  will  be  joint  and  combined  with 
participation  from  NATO,  USAREUR,  USAF6,  and  NAVEUR.  It  will  involve  the  Warrior 
Preparation  Center  (WPC),  a  Joint  Task  Force,  Allied  Headquarters,  and  Exercise  Control  Center 
in  Germany.  The  WPC  will  be  running  an  ALSP  supported  exercise  at  the  Joint  Staff  level.  The 
Army  will  have  instrumented  forces  at  Hohenfels  linked  with  virtual  simulators  at  another 
location  in  Germany.  The  scenario  uses  France  as  an  island  nation  with  live  forces  operating  in 
Germany  with  virtual  and  constructive  forces  in  France.  The  Navy  will  have  two  carrier 
battlegroups,  one  in  the  South  and  the  other  in  the  North.  The  technology  will  be  supporting  the 
training  objectives  and  after  the  exercise,  3TRI.COM  plans  to  keep  the  technology,  software  and 
communications  infrastructure  in  place. 

From  a  STOW-E  94  Joint  Task  Force  with  10,000  entities  and  15  locations,  ARPA  is  looking  to 
a  USACOM  sponsored  STOW  97  Joint  Theater  with  100,000  entities  and  about  50  locations,  to  a 
STOW  2000+  which  will  be  an  operational  system  with  more  than  100,000  entities  and  more 
than  50  participating  locations  worldwide.  Mnlti-level  security  (MLS)  will  be  a  major  issue 
since  systems  and  forces  will  be  operating  a‘.  various  security  levels  including  unclassified, 
secret,  secret  NO  FORN,  and  NATO.  ARP  A  is  looking  at  how  advanced/new  technology  can 
transition  into  the  operational  world. 

The  major  problem  driving  use  of  simulation  for  readiness  is  the  complexity  of  the  current  and 
future  warfighting  and  peacekeeping  world,  Complexity  demands  more  practice,  prototyping 
and  experimentation  than  the  DoD  budget  can  support.  M&S  can  bridge  the  gap  by  providing 
synthetic  environments  to  expand  training  horizons,  to  develop  a  new  acquisition  paradigm  and 
to  explore  new  technologies.  There  is  a  Senior  Readiness  Oversight  Council  in  which  DDR&E 
is  a  member  and  a  Readiness  Working  Group  in  which  DMSO  participates  The  Defense 
Science  Board  (DSB)  Readiness  Task  Force  also  supported  the  use  of  M&S  for  joint  training. 

There  is  a  new  Joint  Service  common  architecture  project  called  JSIMS.  The  MOA  is  being 
staffed  and  three  Services  have  signed  up  and  are  forming  a  JPO.  The  initial  product  will  be  the 
architecture  for  the  next  generation  of  constructive  wargames.  Currently,  JSIMS  is  funded 
mainly  by  the  Air  Force  and  additional  DMSO  funding. 

The  revised  Science  and  Technology  (S&T)  strategy,  currently  in  draft,  identifies  20  technical 
areas,  one  of  which  is  Modeling  and  Simulation.  Each  area  is  developing  a  Technology  Area 
Plan  (TAP)  that  will  guide  long  term  S&T  investments.  M&S  is  mentioned  in  most  of  the  19 
other  areas  and  the  M&S  plan  will  show  interaction  with  over  50%  of  the  other  technology  areas. 
The  DoD  is  pursuing  simulation  as  a  strategic  technology:  simulate  before  you  build,  simulate 
before  you  buy,  and  simulate  before  you  fight.  These  will  help  the  warfighter  fight  smarter. 
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DMSO  is  currently  working  on  the  Defense  Molding  and  Simulation  Master  Plan.  It  will  ‘bcus 
on  readiness,  be  coordinated  with  the  TAP,  build  a  common  vision  across  the  entire  M&S  range 
of  DoD,  includes  technical  assessments  (one  will  include  the  data  issues  area  of  M&S),  and 
«ction  plans  to  build  interoperability  and  jointness  and  to  fill  voids.  It  will  be  the  basis  for 
investment  strategy  and  will  be  updated  annually.  DMSO  is  changing  its  investment  strategy 
from  a  wide  sprinkling  of  funds  to  focusing  resources  on  a  few  critical  areas.  For  example,  they 
may  be  delegating  $20  million  to  be  spent  between  2-3  projects.  They  will  be  working  with  the 
Services  to  focus  funding  on  interoperability  problems. 

LTC  (P)  Wiedewitsch’s  closing  remarks  were  that  the  new  DMSO  Director  CAPT  Jim 
Hollenbach  does  have  an  understanding  and  background  in  data  and  should  be  “able  to  talk  our 
language.” 

Dr.  Chten  Huo:  Modeling  and  Simulation  Data  Administration 

Dr.  Huo  acts  as  the  Functional  Data  Administrator  (FDAd)  for  M&S  and  is  responsible  for 
defining  and  carrying  out  the  M&S  data  administration  program.  He  is  the  co-chair  of  the  I/DB 
TWG. ' 

He  presented  the  four  DMSO  objectives:  (!)  providing  a  technical  framework  for  M&S;  (2) 
developing  authoritative  representations;  (3)  integrating  live,  virtual  and  constructive 
simulations;  and  (4)  broadening  M&S  applications. 

The  overarching  purpose  of  the  DMSO  data  related  activities  is  to  support  the  DMSO  objectives 
by  promoting  interoperability,  sharing  and  reuse  of  data  and  models.  This  purpose  is  being 
carried  out  in  coordination  and  compliance  with  DISA/JIEO/CIM;  through  data  standardization 
and  related  efforts  not  being  addressed  by  CIM  or  the  commercial  world;  and  with  participation 
and  concurrence  from  the  M&S  community. 

The  M&S  FDAd  responsibilities  include:  implementing  an  M&S  DA  infrastructure  to  establish 
community  consensus  on  policies,  procedures  and  standards;  identifying  and  promulgating  DA 
methodology  and  tools;  addressing  important  technical  issues  (e.g.,  complex  data  standards,  data 
VV&C,  authoritative  data  sources);  establishing  an  M&S  repository  requirement,  directories  for 
databases  and  M&S;  and  facilitating  the  interchange  of  information  and  lessons  learned.  Many 
of  these  activities  are  being  carried  out  through  the  I/DB  Technology  Working  Groups  and  their 
Subgroups:  Complex  Data  (Subgroups:  categorization  and  guidelines,  taxonomy,  pilot  studies), 
Data  VV&C  (Subgroups:  data  VV&C  guidelines  and  quality  profile,  authoritative  data  sources, 
and  pilot  studies);  and  Data  Standards  (Subgroups:  repositories  and  DIS  data  standards). 

3.4  REPORTS  FROM  I/DB  TASK  FORCES  AND  SUBGROUPS 

Mr.  Bob  HartUng:  Report  from  M&S  Data  W&C  Task  Force  Subgroup  on  W&C 
Guidelines  Including  Data  Quality  Profile 

The  Data  VV&C  Task  Force  is  chaired  by  Iris  Kameny.  The  Guidelines  Subgroup  is  co-chaired 
by  Bob  Hartling  (Navy)  and  Mark  Ralston  (Army).  The  Authoritative  Data  Sources  and  Data 
Centers  Subgroup  is  co-chaired  by  Bill  Dunn  (Army)  and  Mike  Hopkins  (CENTCOM). 

The  long  range  objectives  for  the  Task  Force  (April  19,  1994)  are  to  ( 1)  develop  guidelines  for 
Data  VV&C  including  definitions  and  process,  cost  models  and  information,  and  quality  profile 
metadata  definitions;  and  (2)  address  authoritative  data  sources  and  their  responsibilities  and  the 
role  of  M&S  data  centers  between  data  sources  and  simulation  centers. 

Progress:  (1)  VV&C  definitions  have  been  agreed  to;  (2)  work  is  ongoing  in  defining  the  VV&C 
process  in  relation  to  the  M&S  VV&A  process  for  DIS  and  non-DIS;  (3)  the  Subgroup  is 
working  with  an  Army  and  DMSO  funded  VV&A  task  force  and  with  the  DIS  VV&A  Group. 
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The  DIS  VV&A  group  falls  under  the  DIS  Fidelity,  Management  and  Usability  Working  Group. 
The  DIS  VV&A  group  has  accepted  the  top  level  VV&C  process  (developed  by  this  Subgroup) 
integrated  into  the  VV&A  process  diagram  that  was  the  outcome  of  the  last  DIS  Workshop.  The 
Subgroup  has  incorporated  the  VV&C  procedures  in  a  quickplanner  that  references  the  DIS 
VV&A  quickplanner.  When  this  is  acceptable,  a  pilot  study  will  be  run  using  the  process 
defined  in  the  quickplanner.  The  plan  is  to  have  initial  data  VV&C  guidelines  ready  to  present  at 
the  September  1994  DIS  Workshop. 

The  DMSO/Army  funded  VV&A  task  force  dealing  with  non-DIS  distributed  simulations  is  also 
collaborating  with  this  Subgroup  since  the  VV&C  process  may  be  different  for  non-DIS  usage. 
Peggy  Gravitz  is  heading  up  that  project’s  data  consistency  task.  When  the  VV&C  process  is 
defined  and  integrated  into  their  VV&A  process,  they  will  run  tests. 

VV&C  Definitions  (April  19, 1994)  from  Data  VV&C  TF  Meeting  at  IDA 

Producer  Data 

Producer  Data  Verification:  The  use  of  techniques  and  procedures  to  ensure  that  data  meets 
constraints  defined  by  data  standards  and  business  rules  derived  from  process  and  data  modeling. 

Producer  Data  Validation:  The  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  within  stated  criteria  and  assumptions. 

Producer  Data  Certification:  Determination  by  the  data  producer  that  data  have  been  verified  and 
validated. 

User  Data 

User  Data  Verification:  The  use  of  techniques  and  procedures  to  ensure  that  data  meets  user 
specified  constraints  defined  by  data  standards  and  business  rules  derived  from  process  and  data 
modeling,  and  that  data  are  transformed  and  formatted  properly. 

User  Data  Validation:  The  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  as  appropriate  for  use  in  an  intended  M&S. 

User  Data  Certification:  Determination  by  the  application  sponsor  or  designated  agent  that  data 
have  been  verified  and  validated  as  appropriate  for  the  specific  M&S  usage. 

Mr.  BUI  Dunn:  Report  from  M&S  Data  W&C  Task  Force  Subgroup  on  Authoritative 
Data  Sources 

Authoritative  Data  Sources  Subgroup  taskings:  provide  agency  names  and  relationships  for 
authoritative  data  sources;  provide  agency  names  and  responsibilities  of  data  centers;  address 
sharing  and  reuse  of  data  between/among  these  data  sources  and  centers;  and  address 
responsibilities  of  data  customers. 

Accomplishments:  They  have  found  that  no  Service  has  comprehensively  assigned  authoritative 
data  sources.  They  have  prepared  a  draft  document  (19  April  1994)  that  discusses  sources  that 
are  authorized  and  defacto,  lists  sources  identified  by  Services  and  CENTCOM,  discusses 
responsibilities  of  data  center  and  data  customer,  and  notes  that  multiple  source  entries  are 
required  due  to  data  aggregation,  and  security. 

Current/future  work:  They  are  working  on  a  data  source  taxonomy  that  will  be  Service 
extensible;  need  to  populate  taxonomy;  need  guidelines  for  security  and  data  release  authority; 
need  to  evolve  document  to  address  policy  and  management  issues;  and  need  to  address 
maintenance  of  the  Authoritative  Data  Source  directory  by  DMSO/IAC. 
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Mr.  Len  Seligman:  Report  from  M&S  Complex  Data  Task  Force  Subgroup  on 
Categorization 

The  Complex  Data  Categorization  Subgroup  goals  are:  develop  categorization  of  “complex 
data”;  and  to  provide  feedback  to  relevant  groups  on  modeling  of  complex  data  (i.e.,  M&S 
Repository  Subgroup,  Defense  Integrating  Repository  System  (DIRS)). 

Accomplishments  are:  They  have  developed  an  initial  categorization  (April  1994)  and  have 
given  feedback  to  Jeff  Wolfe,  DIS  A/JIEO/CIM/DAPMO,  on  the  preliminary  version  of  the  DIRS 
model  with  respect  to  derived  data. 

Definition  of  complex  data:  Data  which  is  difficult  or  awkward  to  model  using  commonly 
existing  techniques  (i.e.,  IDEF1X  and  other  kinds  of  Entity-Relationship  modeling). 

Non-exhaustive  categories  of  data  meeting  the  definition  of  being  “complex”:  inheritance, 
composition,  derivations,  mappings,  artifacts  of  legacy  systems. 

Next  steps:  need  to  assess  appropriateness  of  different  modeling  techniques  for  complex  data 
(such  as  object-oriented  modeling  techniques  or  IDEFxx);  and  continue  discussions  on  DIRS  and 
other  relevant  repository  efforts. 

Ms.  Iris  Kameny:  Report  on  M&S  Data  Standards  Task  Force  and  also  on  Taxonomies 
The  Data  Standards  TF  has  two  Subgroups:  Repositories  co-chaired  by  Jim  Augins  (support  of 
NAVY/ARMS)  and  Pete  Valentine  (Army/JDBE),  and  DIS  Data  Standards  chaired  by  Walt 
Swindell  (Army/TRAC). 

Suggestions  for  possible  scope  and  focus  of  group  (Feb.  15,  1994)  were:  coordination  of  data 
modeling  and  data  standards  development  (standards  committees,  consortia,  etc.),  M&S 
repositories,  M&S  standard-based  data  libraries  (data  sources),  data  classification  (i.e.,  data 
security  issues). 

The  output  of  the  April  20, 1994  meeting  included  five  issues  in  addition  to  the  formation  of  the 
Repository  Subgroup:  (1)  paper  to  DIS  on  need  for  data  standards  (being  addressed  by  Swindell, 
Haddad  and  Kameny);  (2)  data  model  for  DIS  enumeration  document  (JDBE  doing);  (3)  M&S 
policies  and  data  standards  (Chien  Huo/DASP);  (4)  starter  set  of  data  elements  for  M&S  (Chien 
Huo);  (5)  develop  M&S  corporate  view  of  sharable  databases  and  standards  based  on 
Components’  requirements  (province  of  Authoritative  Data  Sources  Subgroup). 

A  paper  on  “DIS  Need  for  DoD  Data  Standards”  was  collaboratively  written  by  Iris  Kameny, 
Luci  Haddad,  Peter  Valentine,  Jim  Watson  and  Walt  Swindell  and  has  been  accepted  for 
presentation  at  the  September  1994  DIS  Workshop.  It  addresses:  the  evolution  of  data  standards, 
the  DIS  Vision  and  its  void  with  respect  to  use  of  data  standards,  specific  ways  in  which  the  use 
of  data  standards  can  benefit  the  DIS  program  and  exercises,  data  standardization,  a  future  vision 
of  DIS  with  data  standards,  and  DIS  roadmap  to  data  standardization. 

The  Repository  Subgroup  has  identified  a  need  to  define  requirements  for  a  distributed  M&S 
Information  Resources  Repository  System.  It  will  enable  rapid  access,  acquisition  and 
processing  of  quality,  consistent,  valid  data  and  other  information  resources  for  use  by  the  M&S 
community.  Its  architecture  will  be  based  on  a  minimal  set  of  standards,  conventions  and 
common  tools,  to  be  implemented  and  tailored  for  use  by  M&S  organizations,  data  centers,  M&S 
programs,  etc.  and  collectively  accessed  as  a  distributed  repositoiy  system.  A  M&S  IRR  may 
contain  objects  such  as  process  and  data  models,  data  standards,  nomenclature  and  symbology 
standards,  directories  (for  M&S  and  databases),  algorithms,  databases,  M&S,  and  common  and 
specialized  tools. 


-  16- 


Taxonomies:  There  is  a  recognized  immediate  need  for  taxonomies  for  the  database  directory, 
the  M&S  directory,  and  the  authoritative  data  source  categories.  There  is  a  Taxonomy  Subgroup 
under  the  Complex  Data  TF  co-chaired  by  Iris  Kameny  and  Dan  Hogg — but  no  meetings  have 
been  held.  RAND  has  developed  initial  taxonomies  for  the  database  and  M&S  directories  and 
sent  them  around  the  I/DB  community  for  review.  Walt  Swindell  (TRAC)  took  the  RAND 
taxonomy  for  databases  and  integrated  it  with  the  TRAC  data  subject  area  taxonomy  and  that  is 
being  used  by  the  Authoritative  Data  Sources  Subgroup.  The  other  taxonomies  have  been  given 
to  Mike  Frame  to  incorporate  into  the  database  and  M&S  directory  data  models.  (As  of  July  14, 
the  Taxonomy  Subgroup  has  been  stopped  and  its  activities  subsumed  by  the  Repository 
Subgroup.) 

Dr.  Mike  Frame:  Report  on  M&S  Directory  and  Database  Directory  Progress 

IDA  has  completed  the  logical  database  design  and  initial  physical  database  design  and 
implementation.  They  are  currently  working  on  refining  the  physical  DB  implementation; 
evaluating  and  processing  Army  Mosaic  data  for  initial  population  of  the  M&S  directory; 
designing  paper  data  capture  “forms”;  evaluating  electronic  data  forms;  and  evaluating  World 
Wide  Web  (WWW)  Mosaic  as  a  browsing  and  query  tool.  The  systems  should  be  in  production 
early  in  1995. 

They  need  to:  complete  the  design  of  the  paper  data  form;  complete  evaluation  of  file  format 
alternatives  (including  those  for  updating  through  a  tagged  exchange  file  standard);  complete 
evaluation  of  WWW  Mosaic;  acquire  fist  of  standard  queries  and  reports;  develop  set  of 
browsing  plans;  be  responsive  to  ARMS  Project  need  for  database  information;  address 
population  of  database  directory;  and  need  policy  and  procedures  for  entry  of  data  into  the 
directories,  and  to  specify  who  does  the  data  editing  and  reviewing,  etc.  DMSO  would  like  each 
Service  to  be  responsible  for  its  own  information. 

Mr.  Howard  Haeker:  Report  on  MORS  Simulation  Data  and  Its  Management 
(SIMDATAM)  Senior  Advisory  Group  (SAG) 

The  SIMDATAM  SAG  was  fottned  by  the  MORS  Executive  Council  whose  chair  is  Mike 
Bauman.  Their  mission  is  to  guide  future  MORS  activities  in  this  subject  area.  Their  purpose  is 
to  provide  guidance,  answer  questions  to  the  MORS  sponsors  and  be  a  sounding  board  on  how 
the  future  SIMDATAM  meetings  should  be  conducted. 

The  first  SIMDATAM  meeting  was  held  on  June  8  in  Colorado  Springs  during  the  62nd  MORS 
Symposium.  An  outcome  of  the  meeting  was  to  make  tentative  plans  for  the  next  SIMDAT 
Symposium  to  be  held  in  M  rch  1995  at  the  Army  War  College.  Plenary  topics  include: 
DDRS/DIRS,  IDEF,  security  rules/regulations.  Chuck  Getty  is  the  point  of  contact  for  the 
plenary  sessions  at  717-732-9210.  Suggested  working  sessions  are:  VV&C;  standards; 
enabling  technologies  (e.g.  research  issues,  storage,  COTS,  object-oriented,  BLOBs,  use  of 
legacy  systems);  reconciliation  of  data  standards  with  major  programs;  and  security.  Security  is 
a  very  important  issue  and  needs  policies  and  procedures.  A  suggestion  was  made  to  work  with 
the  DIS  Security  Council;  Chris  Bowens  was  suggested  as  a  POC  at  717-732-9210. 
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3.5  DATA  ADMINISTRATION,  STANDARDIZATION  AND  MODELING  ACTIVITIES 

Mr.  Bob  Molter:  Update  on  ASD(C3I)  Data  Standardization  Policies  and  Procedures 
Molter  went  over  the  DoD  Data  Administration  program  implementation  status  with  regards  to 
policy,  standards,  infrastructure,  procedures,  tools,  DA  training,  and  miscellaneous.  Policy  is 
incorporated  in  8320.1  approved  September  26,  1991. 

Standards:  Database  language  is  FIPS  127-2  with  current  work  on  3.  The  IRDS  FIPS  156  may 
be  replaced  in  the  future  by  combining  IRDS2  and  PCTE.  He  expects  the  Federal  Government 
to  require  the  use  of  IDEF1X  (FIPS  184)  for  data  modeling  and  IDEFO  (FIPS  183)  for  process 
modeling  by  a  year  from  now.  Currently,  DoD  8020.1  says  to  use  IDEFO  for  process  modeling. 

Infrastructure:  The  people  are  in  place  and  megacenters  are  being  planned.  We  asked  some 
questions  about  the  megacenters  as  they  appear  to  have  much  in  common  with  needs  of  M&S 
data  centers  and  he  suggested  we  get  a  brief  at  the  next  meeting  from  Bill  Tuffy — particularly  on 
their  concept/use  of  client/server  architecture.  An  enterprise  integration  implementation  strategy 
was  signed  last  month.  They  are  planning  to  use  EDI  (electronic  data  interchange)  and  to  get 
data  out  of  documents  through  use  of  an  SGML  to  tag  parts  of  the  document. 

Procedures:  Data  Administration  (DoD  8320. 1-M)  was  approved  March  1994;  Data  Element 
Standardization  (DoD  8320.1-M-l)  was  approved  January  1993,  and  data  modeling,  data 
security,  data  quality  assurance  and  database  administration  are  in  draft.  Jerry  Cooper  is  the 
POC  for  security  and  data  quality  (703-636-6900).  DoD  8320.1-M-x  on  data  modeling  will  be 
coming  out  soon  for  review.  They  are  trying  to  accelerate  data  element  standardization  by 
looking  at  subfunctional  areas  with  all  relevant  parties  joining  in  on  collaborative  sessions.  For 
the  spatial  data,  they  are  having  collaborative  sessions  but  addressing  well  defined  subsets  such 
as  harbors,  transportation  focus  on  systems,  etc. 

Tools:  Enterprise  model  published  January  1994  includes  the  DoD  strategic  data  and  process 
models.  He  drew  a  pyramid  with  the  Enterprise  Model  at  top  and  part  way  down  the  activity  and 
data  models  for  strategy  split  apart,  then  there  was  a  horizontal  split  and  below  the  Enterprise 
Activity  Model  was  the  DoD  activity  model  and  below  the  Enterprise  Data  Model  was  the  DoD 
Data  Model.  He  said  the  strategic  level  view  of  both  the  activity  and  the  data  model  are  called 
the  Enterprise  Model  and  below  is  the  DoD  Activity  Model  and  the  DoD  Data  Model.  Upon 
questioning,  he  said  that  the  C2  Core  Model  is  an  external  view  of  the  DoD  Data  Model:  e.g., 

100  entities  from  the  C2  Activity  Model.  He  said  to  talk  to  Phil  Cykana  about  integration  of  the 
C2  core  model  with  the  DoD  Data  Model.  There  is  a  need  to  map  these  so  that  when  the  C2 
Core  Model  changes  it  is  reflected  in  the  DoD  Data  Model  and  vice  versa.  Commercial  products 
are  needed  to  help  with  the  DoD  Data  Model  both  in  viewing  it  and  in  growing  it.  The  DAPMO 
is  looking  for  some  way  to  distribute  the  DoD  Data  Model. 

The  central  repository  (DDRS)  is  operational  and  populated  with  standard,  developmental  and 
migration  data.  He  is  concerned  with  why  the  M&S  community  is  developing  an  M&S 
repository  architecture.  DA  is  in  the  process  of  selecting  candidates  for  a  migration  system. 
They  have  data  repository  agreement  on  the  DDRS;  there  is  an  interim  repository.  The  DDRS 
solution  includes  acceptance  of  a  PC  companion.  ICASE  was  awarded  in  1994. 

Miscellaneous  included:  2  DASPs  have  been  published,  the  FY94  Planning  Guidance  was 
published,  and  a  data  migration  prototype  using  reverse  engineering  is  ongoing,  data  migration 
and  implementation  planning  has  begun  and  they  are  selecting  a  migration  system. 
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Mr.  Bruce  Rosen:  Status  of  Repository  Standards 
The  scorecard  for  ANSI  X3H4: 

X3. 138-1988,  IRDS  standard  for  user  interface  command  language  for  screen/panel 
interface 

X3. 195-1991,  IRDS  export/import  file  format  specification  for  bulk  transfer 

X3. 185-1992,  IRDS  services  interface  (so  other  software  can  use  IRDS  database): 
supports  entity/attribute  relationship  model  with  attributes  on  the  relationship 

The  scorecard  for  ISO: 

ISO/IEC  JTC1  (Joint  Technical  Committee),  SC21  WG3  IRDS  Rapporteur  Group 

ISO  IS  10728(E),  IRDS  services  interface:  interface  not  same  as  ANSI 

X3. 185-1 992.  It  is  based  on  different  models  and  beliefs.  The  ISO  standard  is  based  on 

SQL  where  ANSI’S  services  interface  is  based  on  database  administration  concepts.  The 

ISO  entity/attribute/relationship  allows  no  attributes  on  relationships  since  it  is  SQL 

based. 

ISO  IS  10027(E),  IRDS  framework  (US  IRDS  fits  in  this  framework) 

European  Computer  Manufacturers  Association  (ECMA):  has  no  standing  as  a  standards 
organization  but  what  they  try  to  put  out  are  “standards”  (mainly  composed  of  Asians  and 
American:;). 

ECMA  Technical  Committee  33  (TC33) 

ECMA  149,  Portable  Common  Tool  Environment  (PCTE) 

They  have  published  a  book  on  PCTE  “PCTE  Standard  for  Repository.”  PCTE  is  meant  to  be  an 
interchange  standard  for  CASE  tools. 

Rookies: 

ECMA  TC33  Technical  Group  for  Object  Oriented  (TGG<Jy. 

North  American  PCTE  initiative  that  is  now  part  of  the  Object  Management  Group 
(OMG)  and  called  Special  Interest  Group  (SIG)  PCTE. 

OMG  is  a  consortia  and  puts  the  OMG  stamp  of  approval  on  what  they  like  and  try  to  get 
to  product  as  soon  as  possible. 

JTC1/EEC  SC21  WG3  and  ANSI  X3H4  next  step  would  be  development  of  IRDS  1.5  standard, 
current  IS  10728  with  some  00  added.  Projected  completion  of  this  was  July  1995.  Next  step 
after  that  would  be  IRDS  2.0,  with  new  full  “00”  capabilities,  projected  completion  of  this  was 
July  1997. 

ECMA  TC33,  ECMA  149,  PCTE:  a  fast  track  JTC1  ballot  of  PCTE  led  to  a  new  standard  for 
repository.  ECMA  requested  that  the  standard  be  placed  under  SC22.  The  IRDS  1.5  and  2.0 
have  been  canceled  in  favor  of  ECMA  TC33  ECMA  149,  PCTE. 

X3H4  has  asked  the  OMC  to  remove  them  as  TAG  to  ISO  IRDS  Rapporteur  Group  and  be 
reassigned  as  TAG  to  the  new  ISO  group  responsible  for  PCTE  and  that  X3H4  be  renamed  to 
whatever  name  is  chosen  for  the  new  JTC1  group  responsible  for  PCTE  standardization. 
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Rosen  believes  that  PCTE  will  become  the  focus  of  the  repository  to  serve  both  software  and 
data  administration  standards  development  and  X3H4  will  end  up  TAG  to  PCTE.  The  repository 
would  be  useful  as  a  tool  integrator  as  well  as  a  data  administration  tool.  PCTE  uses  an  SDS 
description  set,  whereby  each  tool  has  its  own  SDS  view  into  PCTE.  CDIF  fits  in  as  a  way  to 
provide  SDS  information.  Some  issues:  PCTE  would  be  the  underlying  mechanism  for  tying 
together  tools  from  various  vendors  by  providing  database  functionality.  However,  PCTE 
doesn’t  assign  unique  IDs  to  objects  only  link  addresses.  There  is  no  concept  of  object  identity. 
Right  now  IRDS  doesn’t  map  into  PCTE  at  all. 

This  could  mean  that  US  IRDS  (X3. 138)  would  remain  viable  as  user  interface  standard  and  US 
IRDS  Export/Import  could  remain  viable  but  US  IRDS  Services  interface  and  ISO  IRDS  (IS 
10728)  would  be  dropped. 

Rosen  says  that  it  is  likely  that  the  ISO  IRDS  group  will  die.  Over  the  last  year  or  so  they  mostly 
consisted  of  representatives  from  the  US,  UK,  Japan,  Australia,  and  a  few  others.  The  UK 
standards  committee  has  stopped  funding  UK  participation. 

As  regards  IRDS  FIPS  156,  government  regulations  say  that  any  organization  buying  a 
dictionary  has  to  buy  a  product  compliant  with  FIPS  156.  Currently,  there  are  two  vendors  sort 
of  compliant  with  IRDS:  INFORMIX  and  INFOSPAN. 

Mr.  George  Endicott:  Report  for  Intelligence  FDAd 
This  scheduled  presentation  was  not  made. 

Mr.  Gary  Hurd:  Report  from  Acquisition  and  Technology  FDAd 
Dr.  Leland  Jordan  is  the  acting  A&T  FDAd.  Mr.  Gary  Hurd  effectively  operates  the  CIM  and 
DA  programs  for  Dr.  Leland  Jordan.  One  of  A&T’s  functional  areas  is  Science  and  Technology 
under  which  M&S  falls. 

The  A&T  CIM  integration  (ACI)  program  was  instigated  by  C3I  and  executed  by  the  ACI  office. 
Its  mission  was  to  facilitate  the  integration  of  all  A&T  CIMS  with  one  another  and  with  non 
A&T  CIM  efforts.  The  A&T  community  is  large.  The  functional  CIM  offices  in  OUSD(A&T) 
are  logistics,  procurement,  environmental  security,  economic  security,  systems  acquisition 
management,  science  and  technology,  test  and  evaluation,  and  atomic  energy.  The  first  three 
have  been  around  longer  and  are  further  along  in  their  programs  than  the  last  five.  The  purpose 
of  the  ACI  Program  is  to  define  an  approach  for  improving  and  integrating  acquisition  policies, 
business  processes,  and  supporting  information  systems;  assist  DoD  officials  in  acquisition 
reform;  and  achieve  near-term  changes  in  how  DoD  operates.  Program  responsibility  resides 
with  functionals  and  is  executed  through  the  functional  stewards.  The  ACI  facilitates  the 
program  start  ups  and  integration  activities. 

The  FY93-94  CIM  initiative  included  implementation  plans  for  the  first  three  areas  of  logistics, 
procurement,  and  environment  and  the  other  five  areas  (added  in  October  1994  by  the  Perry 
memo)  have  been  engaged  in  defining  charters,  organizational  structure  and  holding  workshops. 
For  FY94,  there  is  an  integrated  DASP  for  the  A&T  community.  A  decomposition  of  the 
Enterprise  process  and  data  models  were  used  as  a  framework  for  integrating  procurement 
models  into  the  Enterprise  Model.  An  information  infrastructure  was  composed  of  a  TAFIM 
based  infrastructure  and  shared  data  topology.  The  shared  information  infrastructure  is  necessary 
in  order  to  migrate  bottom-up  data  standards  into  the  top-down  enterprise  standards.  However, 
their  bottom-up  models  didn’t  integrate  so  well  with  the  Enterprise  Model.  He  invited  Chien 
Huo  to  talk  to  him  more  about  what  they  did. 
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Expected  benefits  are:  improvement  and  control  of  A&T  processes;  identification  of  targets  for 
performance  improvements;  basis  for  improved  information  systems;  near-term  reductions  in 
information  systems  costs;  and  near-term  improvements  in  decision  support  systems  information. 

Mr.  Stan  Plummer:  Report  from  C2  FDAd 

Ms.  Deborah  Castleman,  OSD  Deputy  Assistance  Secretary  of  Defense  for  C3,  is  the  C2  FDAd. 
The  C2  FDAd  mission  is  to  achieve  a  fully  interoperable  C2  environment  through  effective  data 
standards  coordination  and  program  development,  including  data  elements  and  data  models  for 
C2  projects,  programs,  and  migration  systems.  The  goals  are  to:  support  Joint  and  Combined 
operations,  aggressively  implement  C2  DA,  and  to  promote  coordination,  cooperation,  and 
participation  of  Functioral  Areas  and  Components.  There  is  a  C2  DASP  for  the  years  1994- 
2001.  The  main  functions  of  the  C2  FDAd  program  are  to:  provide  standard  data  elements, 
maintain  the  C2  core  data  model,  integrate  C2  subarea  data  models,  and  coordinate  with  FDAds 
and  CDAds.  The  program  also  is  supporting  the  Global  Command  and  Control  System  (GCCS), 
which  is  its  principal  C2  migration  system,  by  accelerating  data  standardization  through  focused 
sessions  and  collaborative  modeling.  This  is  a  Quickfix  migration  system  intended  to  replace 
WWMCCS  and  will  include  a  Joint  Service  agreed  to  Common  Operating  Environment  (COE). 
COL  Connelly  is  in  charge  of  GCCS  integration. 

In  the  data  modeling  area  they  are  developing  subarea  models  that  will  be  integrated  with  the  C2 
Core  Model  and  as  a  result  may  propose  changes  to  the  DoD  Data  Model.  Ongoing  efforts 
include:  C2  Core  Data  Model,  Fire  Support  Model,  Air  Operations  Model  (in  candidate  status), 
and  SOCOM  Model  (Logicon  will  be  working  on  integration  of  this  with  C2  Core  Model  later 
this  summer).  The  C2  data  elements  have  been  gathered  from  the  C2  Core  Data  Model,  C2  input 
to  starter  set  (exists  but  not  as  a  result  of  data  modeling),  and  JUD1  data  elements  (on  hold  for 
now). 


C2  Core  Data  Model  status  (8  July  1994): 


Prime  Words 

Data  Elements 

Developmental 

103 

349 

Candidate 

31 

189 

DoD  Standards 

53 

30 

The  C2  FDAd  has  recognized  the  configuration  management  of  data  models  to  be  a  problem. 

Ms.  Carla  Von  Bernewitz:  JIEO/CIM  Plan  for  M&S  Support 

Ms.  Von  Bernewitz  is  the  Director  of  the  Data  Administration  Program  Management  Office 
(DAPMO).  The  DAPMO  mission  is  to  define,  plan  and  organize  the  DoD  Data  Administration 
Program  to  promote  definition,  organization,  operation,  supervision,  and  protection  of  data 
within  the  DoD  as  a  strategic  resource.  Originally  DoD  had  an  8  year  schedule  for  standardizing 
data  but  Perry’s  Oct.  13,  1993  memo  mandated  a  3  year  requirement  for  standard  data  and  so  an 
interim  solution  was  needed.  The  interim  solution  is  to  develop  interim  standards  including 
industry  standards  and  map  existing  databases  to  those  standards  by  using  a  different  approach 
than  strictly  top-down.  The  long  term  solution  is  integrated  databases  that  use  the  data  standards. 
Data  as  part  of  the  system  life  cycle  was  shown  in  three  steps:  (1)  the  plan  for  data 
standardization  is  developed  and  described  in  the  DASP;  (2)  data  standardization  is  carried  out 
based  on  DoD  8320  and  results  in  metadata;  and  (3)  the  data  standards  are  used  by  taking  subject 
databases  and  mission  applications  and  re-engineering  AIS  applications  based  on  standards 
through  use  of  standard-based  data  (DAPMO),  applications  (PSA/FAPM),  and  technical 
infrastructure  (DISO).  The  AIS  developments  use  the  DDRS  and  I-CASE  tools 

The  standardization  review  process  consists  of  informal  review  (developmental)  followed  by 
formal  review  (candidate)  based  on  formal  review  packages  that  receive  cross-functional  review, 
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followed  by  approved  status  (standards  stored  in  DDRS)  or  disapproved  or  issues  are  forwarded 
for  resolution  to  the  DASD(IM).  Collaborative  modeling  involves  joint  modeling  and  review  of 
a  subject  area  with  fdads,  FDAds,  CDAds  and  DAPMO  all  participating  to  get  to  standard  data 
elements  more  quickly.  This  is  a  change  from  looking  at  legacy  systems  for  possible  data 
elements  to  standardize.  Instead  the  process  is  to  select  subject  areas  to  data  model  and  from 
those  data  models  define  the  data  standards.  Data  standards  are  not  being  hard  coded  rather  they 
are  expected  to  change.  Washington  defines  the  tables  of  prime  words  and  class  words  and  then 
lets  the  collaborative  efforts  and  benchmarks  use  those  as  parts  of  the  data  element  names  rather 
than  the  earlier  strictly  top-down  naming  approach.  They  need  CASE  tools  to  aid  in  the 
collaborative  modeling. 

The  DDRS  was  recently  put  on  an  KP  box  so  there  are  no  more  problems  with  access.  It  will  use 
an  Army  tool  for  downloading  and  will  also  be  distributed  re  CD-ROM  disk.  Data  models  will 
be  available  by  the  end  of  July.  They  need  a  repository  solution  to  tie  all  the  standardization 
objects  together.  The  DDRS  bulletin  board  has  60  internal  customers,  17  FDAds  and  29  CDAds. 
Chien  Huo  should  be  on  the  DDRS  steering  committee  since  M&S  is  interested  in  a  repository. 

The  data  standardization  status  (July  8, 1994): 

Prime  Words  Data  Elements 


Candidate  76  304 

Approved  280  632 

Comment  was  that  they  haven’t  seen  use  of  the  data  standards  by  application  developers  yet. 

And  a  question  was  left  open  about  moving  to  object-  oriented,  and  if  so,  then  when. 

Mrs.  Melody  Rood:  Report  on  ISO  Data  Representation 

Her  purpose  was  to  familiarize  us  with  the  emerging  international  ISO  standard  for  data 
elements,  ISO/IEC  1 1 179  “Specification  and  Standardization  of  Data  Elements.” 

The  purpose  of  the  standard  is  to  enhance  sharing  of  data  (meaning/content)  across  systems  by 
standardizing  data  element  design.  A  common  understanding  of  data  will  be  achieved  by 
standardizing  the  attributes  of  data  elements  (metadata)  to  fully  specify  a  data  element’s 
meaning,  identification  and  representation,  and  by  standardizing  terminology. 

ISO  and  ANSI  standards  committees  in  this  area:  ISO/IEC  JTC1/SC14  Data  Element  Principles 
(International  Organization  for  Standardization/Intemational  Electrotechnical  Commission,  Joint 
Technical  Committee  1,  Subcommittee  14);  and  ANSI  X3L8  Data  Element  Representation. 

Important  things  to  know  about  ISO/IEC  11179:  it  doesn’t  standardize  individual  data  elements; 
is  methodology  and  application  independent;  developed  with  intemational/multilingual  users  in 
mind;  and  has  many  testers  including  Bellcore  and  EPA  in  US. 

Outline  of  ISO/IEC  1 1 179,  Data  Element  Specification  and  Standardization,  Parts  1-6: 

Part  1,  Framework  for  the  Specification  and  Standardization  of  Data  Elements 

Part  2,  Classification  of  Concepts  for  the  Identification  of  Domains 

Part  3,  Basic  Attributes  of  Data  Elements:  five  kinds  are  identifying,  definitional,  relational, 
representational,  and  administrative  (these  total  to  less  than  50  metadata  elements) 


Part  4,  Rules  and  Guidance  for  the  Formulation  of  Data  Definitions 
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Part  5,  Naming  and  Identification  Principles  for  Data  Elements:  a  unique  nonintelligent 
identifier  must  be  assigned  to  each  data  element 

Part  6,  Registration  of  Data  Elements 

(More  information  on  each  of  these  Parts  can  be  found  in  the  briefing  charts.) 

Inputs/comments  to  the  standard  are  welcome  via  ANSI  X3L8  or  via  Mike  Gorman  to 
mgorman@mitre.org. 

What  is  a  data  element?  It  is  a  single  unit  of  data  that  in  a  certain  context  is  considered 
indivisible.  It  is  a  field  in  relational  database  tables,  attribute  of  entity  type  in  Entity- 
Relationship  Diagram  (ERD),  attribute  in  logical  data  model,  and  attribute  of  object  class  in 
object  model. 

Phil  Cykana:  Discussion  of  Initiatives  on  External  Data  Standards  and  DoD  Data 
Administration 

8320. 1  series  guidance  includes  national,  international  and  federal  standards.  There  is  a  need  to 
accelerate  development/establishment  of  DoD  data  standards.  He  showed  us  the  approach  used 
in  taking  FIPS  standards  (FIPS  PUB  5-2  codes  for  identification  of  states,  DC  and  outlying  areas 
of  US  and  associated  areas,  and  FIPS  PUB  6-4  which  covers  counties  and  equivalent  entities  in 
US)  and  integrating  these  into  an  IDEF1X  model. 

He  showed  us  an  example  of  the  Defense  Logistics  Agency  interfacing  ANSI  X.12  terms  and 
definitions  into  DoD  data  standards.  The  EDEF1X  metamodel  for  this  shows  the  relationships  of 
ANSI  X.  12  transaction  set,  data  segment  and  data  element  to  an  extemal-data-element  entity  that 
is  related  to  a  DoD-standard-data-element  entity. 

The  expected  results  of  this  are:  use  of  1500+  X.12  data  standards  by  DoD,  FDAd  and  CDAd 
participation;  and  acceleration  and  use  of  DoD  data  standards. 

3.6  REPORTS  ABOUT  OTHER  ORGANIZATIONS 

Ms.  Patricia  Liggett:  Report  on  NASA  Conference  on  Catalog  Interoperability/NASA 
Science  Internet  (CI/NSI) 

NASA  Distributed  Active  Archive  Centers  (DAACs)  are  single  points  of  contact  for  a  user  to  get 
data  from  other  sites  (DAACs).  NASA  would  like  all  of  the  DAACs’  data  to  be  based  on  the 
same  data  model.  They  are  moving  toward  the  HDF  standard  but  don’t  want  to  restrict  the 
NASA  EOS  DIS  world  to  a  single  standard.  Right  now  they  have  r  o  firm  requirements, 
“centers”  vary  from  very  large  DAACs  to  a  single  scientist  working  in  the  boondocks  collecting 
data.  They  need  to  have  a  system  that  makes  it  easy  for  the  user  to  access  and  acquire  data  and 
for  centers  to  supply  data. 

As  far  as  data  quality:  scientific  data  may  change  as  the  type  of  processing  applied  to  it  changes 
and  there  may  even  be  a  need  to  go  back  to  source  data  and  re-derive  it  using  new  methods — a 
very  costly  thing  to  do.  They  have  issued  a  paper  on  data  quality  and  security.  It  is  easier  to 
exert  quality  and  security  measures  on  the  large  DAACs  than  on  the  smaller  collector.  Some 
NASA  data  that  is  available  within  hours  to  days  for  use  in  disasters  is  weather  and  global 
change  data. 

She  believes  the  DAACs  need  to  talk  to  DoD  about  the  DoD  concept  of  megacenters.  The 
DAACs  will  be  used  worldwide.  The  Europeans  and  Japanese  are  cooperative  partners  in 
development  and  use  of  international  space  centers. 
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I/DB  members  can  get  on  the  NASA  mailing  list  for  RFPs  resulting  from  Cooperative 
Agreement  Notices  (e.g.,  between  ARPA  and  NASA).  The  subject  of  the  last  one  was  on 
applying  NASA  data  to  other  uses. 

They  have  been  very  concerned  with  catalog  interoperability  (Cl)  and  ways  to  index  and  present 
data.  They  now  use  SQL-based  databases  but  they  are  looking  at  object-oriented  DBMS. 

The  Committee  on  Earth  Observation  Satellites  (CEOS)  has  an  International  Directory  Network 
(IDN).  They  are  predicting  saturation  of  the  network  due  to  scientific  users  and  need  better 
compression  techniques  and  network  management  techniques.  Their  directories  are  DIF  based 
which  takes  time  and  resources  to  implement.  Many  archives  already  make  their  information 
available  through  Gopher  and  WWW,  etc.  and  some  form  of  cross-archive  (DIF  and  non-DIF 
based)  text  searches  in  pre-selected  areas  of  interest  would  be  a  useful  service  (with  warnings  to 
the  user). 

The  EOSDIS  Core  System  (ECS)  Data  Handling  System  (EDHS)  is  the  on-line  distribution  and 
storage  system  for  documents  about  ECS.  It  is  maintained  by  Hughes  Applied  Information 
Systems  at  the  ECS  development  facility  and  a  list  of  relevant  white  papers  and  technical  papers 
can  be  found  in  the  briefing  section  of  this  proceedings. 

Mr.  Allen  Hess:  BMDO  Tests/Experiments  Data  Management  Lessons  Learned 
Allen  Hess  described  what  BMDO  did  to  provide  a  foundation  for  data  management.  This 
included  establishing  test  data  centers,  consolidating  all  test/experiment  data,  creating  a  Data 
Centers  Standards  Committee  (DSCS)  and  a  Phenomenology  Science  and  Analysis  Group  User 
Products  Information  Group  (PSAG/UPIG);  drafting  a  data  management  directive  and  vision, 
and  providing  core  funding  to  support  data  management  activities.  All  of  the  above  would  be 
useful  within  the  M&S  community. 

The  data  centers  are  the  focus  of  data  management  and  not  only  serve  as  primary  archives  but 
also  facilitate  analycis  by  providing  timely  access  to  well  documented  and  validated  data  and 
data  products.  They  also  satisfy  federal  regulations  requiring  preservation  of  data  collected  from 
federally  funded  projects. 

The  briefing  (found  in  the  briefing  section  of  this  proceedings)  goes  into  detail  on  data  center 
lessons  learned  and  should  be  reviewed  by  the  I/DBTWGs  and  Subgroups.  In  particular  the 
Repository  Subgroup  should  look  at  their  industry/federal  standards  and  distributed 
DBMS/client-server  capabilities.  The  FDAd  could  see  how  they  ensure  the  use  of  data  standards 
across  data  centers.  The  Authoritative  Data  Sources  Subgroup  may  be  interested  in  the 
establishment  of  policy  for  the  management  of  all  the  data  (e.g.,  responsibilities),  how  release 
authority  is  handled,  and  how  to  prevent  duplication  of  data  among  data  centers.  A  Security  TF 
would  pay  attention  to  security  issues  and  user  support  and  classification  of  aggregate  data.  The 
Taxonomy  Subgroup  may  be  interested  in  their  robust  keywork  list  for  use  with  summary 
catalogs  and  database  searches  (at  least  as  a  possible  subtree  for  BMDO  type  of  data).  The  Data 
VV&C  TF  may  be  interested  in  interacting  with  them  on  what  they  are  doing  in  the  VV&A/C 
areas. 

Hess  provided  us  with  a  good  graphic  of  the  management  of  experimental  and  modeled 
phenomenological  data  (see  brief).  He  points  out  that  there  are  lots  of  DoD  and  federal 
regulations  related  to  data  centers  that  need  to  be  observed.  He  said  that  BMDO  developed  a 
data  cost  model  but  it  is  not  being  used  due  to  cutbacks  (we  might  want  to  look  into  this  more). 
To  access  BMDO  data,  one  just  needs  to  get  to  one  data  center  since  the  master  catalog  can  be 
accessed  from  any  of  them. 
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Dr.  Robert  Walker:  Overview  of  C2  Core  Data  Model 

The  purpose  of  data  models  is  to  provide  a  high-level  specification  of  the  information  inputs  and 
outputs  of  functional  processes  and  information  items  subject  to  change,  and  to  provide  a 
consistent  basis  for  data  element  standardization.  Data  models  do  not  limit  the  choice  of 
representations  for  storing  data  physically  in  a  system,  presenting  data  to  users,  presenting  data  to 
communications  systems,  or  exchanging  data  internally  within  an  automated  system.  They  do 
not  limit  the  choice  of  language  for  users,  programmers  or  database  query.  He  showed 
Zachman’s  Framework  for  information  systems  architecture  and  related  data  modeling  to  data, 
activity  modeling  to  function,  and  DISA  architecture  to  the  network. 

The  DoD  Enterprise  Model  provides  a  defense  enterprise-level  view  of  the  DoD  Data  Model. 

The  DoD  Data  Model  is  a  single  integrated  data  model  for  DoD  and  consists  of  approved  entities 
and  attributes  under  8320. 1-M- 1.  The  C2  Core  Model  is  a  C2  view  of  the  DoD  Data  Model. 

The  JIEO/DAPMO  integrates  C2  and  other  functional  area  data  models  into  the  DoD  Data 
Model. 

The  C2  Core  Model  was  mandated  by  the  MCEB  to  be  finished  by  8/93  and  revised  in  7-8/94.  It 
differs  from  the  C2  Generic  Core  Model  accepted  by  NATO  in  that  it  excludes  object-item  and 
object-type  to  simplify  integration  with  the  DoD  Data  Model.  It  has  been  recommended  for  use 
by  the  C2  FDAd,  Intel  FDAd  and  Army  ODISC4.  It  serves  as  a  core  for  C2  model  development 
and  integration. 

Concepts  underlying  the  C2  Core  Data  Model:  that  the  C2  fccus  should  be  on  data  for  the 
elements  of  the  battlefield  and  their  employment,  and  that  there  is  a  common  core  underlying  the 
C2  functional  area.  Battlefield  elements  comprise:  person,  unit,  material,  feature  and  facility. 
Activities  employ  objects  both  as  resources  and  as  objectives  and  the  objects  occur  both 
genetically  by  type  and  specifically  by  item.  Objects  can  be  located  in  a  single  way.  Activities 
can  be  grouped  and  structured  as  actions  to  specify  subactions,  plans,  orders  and  requests,  and 
events.  The  C2  Core  Data  Model  can  provide  a  basis  for  integrating  subfunctional  areas,  e.g.,  the 
Fire  Support  Data  Model  is  the  first  extension  of  the  common  core. 

Challenges  are  to  coordinate  the  C3I  modeling  efforts  by  bringing  together  M&S  with  C2  and 
intelligence  initiatives;  integrating  communications-electronics  and  environmental  data 
modeling;  expanding  C2  as  new  requirements  emerge;  integrating  C2  into  one  C2  data  model; 
supporting  development  of  the  GCCS  data  model;  and  accelerating  model  integration  and  data 
standardization.  Miscellaneous  observations  include:  GCCS  bolted  together  JOPES  and 
Service’s  data  with  no  integrated  data  structures;  8320  is  missing  standards  for  bit  encoding;  and 
the  C2  Core  Data  Model  development  is  missing  guidance,  funding,  real  estimates  and 
agreement. 

Mr.  Peter  Valentine:  Report  on  IEEE  1320.2  IDEF1X  Working  Group 
IEEE  1320.2  was  formed  from  the  group  that  wrote  the  current  FIPS  184  and  is  the  WG 
attempting  to  define  the  next  generation  of  IDEF1X  language.  It  is  a  mix  of  industry  and 
government  representatives. 

List  of  Requests  for  Changes  (RFCs)  that  the  WG  is  working  on: 

•  Make  the  FIPS  184  formalization  description  more  understandable 

•  Add  Rule  Constraint  Language  (RCL)  and  required  support 

•  Allow  Discriminator  from  ancestor  (allow  use  of  any  attribute  in  view  as 
Discriminator) 

•  Support  Multiple  Inheritance 
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•  Re-examine  Alias  support 

•  Specify  the  Transform  Model 

•  Agree  on  Interchange  format:  IDL  PAR  withdrawn,  Case  Data  Interchange  Format 
(CDIF)  under  discussion 

•  Dictionary  Hierarchy:  hierarchies  implicit  in  domains 

•  Usage  Guidelines  for  Upper  Zachman  Framework  rows:  higher  level  modeling 
guidance 

•  Full  method  support:  includes  support  for  ADTs,  methods  with  arguments  and 
specification  language 

Other  efforts  include:  conformance  guidelines  being  worked  on  by  NIST  (Bruce  Rosen);  and 
WG  members  currently  evaluating  RCL  by  creating  examples  in  each  of  their  areas  of  expertise. 

Issues:  lack  of  vendor  participation,  and  funding  for  development  language  components. 

Peter  noted  that  the  IDEFO  WG  has  better  representation  from  vendors  than  does  IDEF1X. 

Peter  believes  KBSI  owns  the  rest  of  the  IDEF  methods:  IDEF3  for  active  processes  and  IDEF4 
for  00. 

The  IEEE  1320.2  is  expanding  IDEF  IX  from  data  modeling  into  object  modeling.  They  are 
planning  to  introduce  layered  standards.  They  have  been  working  with  NIST  on  the  IDEF1X 
conformance  guidelines.  Next  Working  Meeting  was  heid  at  NIST  on  August  7-9.  The  POC 
is  Mary  Laamanen  301-975-3260. 

Mr.  Jim  Glymph:  The  IDEF1X  Experience 

The  Army  data  standardization  organization  is  moving  to  DISA/JIEO/CIM  on  1  October  1994. 
Carla  Von  Bemewitz,  Director  of  the  DAPMO,  will  be  their  new  boss.  They  will  be  part  of  DoD 
but  will  be  working  for  the  Army  on  the  Army  backlog  until  it  is  finished.  The  Army  Data 
Dictionary  now  uses  the  DoD  8320.1-M-l  structure,  uses  DoD  class  words,  and  uses  “person” 
rather  than  “individual”.  He  said  that  AR  25-9  will  be  going  away  and  that  when  they  move  to 
DISA,  the  Army  data  standardization  process  will  be  to  submit  candidate  proposal  packages  to 
Army  ODISC4  where  they  will  be  submitted  to  the  appropriate  FDAd. 

They  have  been  working  in  the  IDEF  IX  modeling  arena  since  Oct  1990  and  have  developed  20 
Army  functional  area  models,  the  Army  data  model,  6  Sustaining  Base  Information  Services 
(SBIS)  application  models  and  integrated  view,  and  reworked  the  battlefield  logistics  model. 

The  Army  Data  Model  consists  of  models  from  TRADOC,  Reserve  component,  SBIS,  finance 
and  accounting  services,  C2  Core  Data  Model,  and  battlefield  logistics.  A  success  story  is  that 
on  modeling  a  recent  training  application  they  got  20%  reuse  of  standard  data  elements  and  40% 
reuse  of  entities.  He  expects  these  reuse  numbers  to  go  up  as  more  areas  are  standardized. 

Glymph  showed  four  different  ways  to  get  to  IDEF IX.  The  preferred  way  is  through  IDEFO 
process  modeling  (they  did  20  functional  models  that  way).  The  process  model  is  developed  to 
the  point  where  one  can  identify  the  information  exchange  requirements.  Another  approach, 
used  for  the  six  SBIS  and  training  models,  is  to  use  the  functional  description  for  the  “to  be”.  A 
third  way  is  a  facilitated  workshop  bottom-up  approach  that  starts  with  a  database  list  of  data 
elements  and  the  user  says  which  he  needs.  The  last  way  is  to  reverse  engineer  a  database  model 
into  IDEF  IX,  which  they  have  done  for  programs  that  reached  MAISRC  without  data  standards. 

Their  lessons  learned:  it  is  critical  to  use  a  facilitator;  need  to  use  the  right  functionals  (customer 
vs  user);  functionals  need  minimum  IDEF  training  (e.g.,  12  hours  not  a  week  or  two);  develop 
standards  as  you  do  the  data  modeling;  and  reuse  data  standards. 


3.7  REPORTS  FROM  SERVICE  M&S  ORGANIZATIONS  WITH  RESPECT  TO  DATA 
RELATED  ACTIVITIES 


Mrs.  Lana  McGlynn:  Army  M&S  Master  Plan 

Purpose  of  the  Army  M&S  Master  Plan  is  to  promote  the  adoption  of  standards  and  common 
tools  and  processes  in  building  and  populating  models  and  simulations  for  use  in  all  applications 
throughout  the  Army.  It  is  required  by  DoDD  5000.59  and  was  published  4  May  1994. 

The  Master  Plan  table  of  contents  is  shown  below  and  discussed  in  more  detail  in  the  briefing 
charts: 


Chapter  I: 

Introduction 

Chapter  II: 

M&S  Environment 

Chapter  III: 

Standards  Development 

Chapter  IV: 

Investment  Strategy 

Chapter  V: 

Plan  Implementation 

Appendix  A: 

Glossary 

Appendix  B: 

References 

Appendix  C: 

Roles  and  Missions 

Appendix  D: 

AMIP/DMSO  Format 

The  Army  will  continue  to  have  a  standalone  data  standards  system,  even  with  Jim  Glymph’s 
group  moving  to  DISA,  but  will  stay  aligned  with  the  functional  areas  and  will  be  merging  more 
and  more  with  them.  They  will  be  supporting  an  online  VV&A  catalog  but  it  will  not  be  open,  it 
will  only  be  available  through  Army  specific  points  of  contact. 

In  Chapter  III,  the  Standards  Development  Process  has  well  defined  tasks:  establish  team 
arrangements;  perform  domain  analysis;  define  standards/  services  required;  develop 
technical/proccdural  standards;  achieve  community  consensus;  build  repositories;  and  educate 
and  assist  modelers/users.  Categories  for  Standards  have  been  defined:  VV&A  methodologies, 
data  standards,  system  services,  environmental  representations,  battlefield  algorithms,  operations 
other  than  war,  strategic  activities,  cost  representation,  distributed  simulation  standards, 
computer  generated  forces,  and  user  interfaces.  Each  category  is  assigned  a  coordinator  and  an 
initial  assessment  is  made  for  each  category  for  the  six  tasks  in  the  standards  development 
process.  A  crosswalk  has  been  done  between  DMSO  TWGs  and  the  corresponding  Army 
standards  categories. 

LtCol  Cheryl  Balombini  and  Maj  Roger  Van  Epps:  Update  front  Air  Force  XOMT 

The  DoD  DA  program  is  driven  by  OSD  Functionals,  supported  by  Components  and  technically 
administered  by  DISA.  It  covers  data  element  standardization,  data  quality,  data  security,  and 
data  base  administration.  In  the  Air  Force  organizational  structure:  LtGen  O’Berry  is  the  CDAd, 
COL  Larry  Sipos  (ret)  is  the  Air  Staff  POC  who  coordinates  with  FDAds,  the  CDAd,  and 
AFC4A/XPSD  whose  POC  is  Mr.  Jim  Neighbors  who  coordinates  with  the  DAPMO,  AF 
functional  data  coordinators,  MAJCOM  DAds  and  others. 
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AF/XOM  provides  guidance  to  the  AF  M&S  community,  oversees  development  activities,  and 
fosters  implementation  of  data  standards  for  M&S  activities.  XOM  functions  are  to  provide  AF 
MS&A  leadership  and  policy;  warfighter  support  for  doctrine,  strategy,  training,  wargaming  and 
exercises;  analysis  support  for  requirements,  force  planning,  COEA,  acquisition,  and  T&E; 
technical  support  for  standards,  VV&A  and  technology  assessment;  and  perform  analysis  to 
support  USAF. 

XOM  is  currently  working  on  a  master  plan  so  there  is  no  timeline  for  the  following 
implementation  steps;  develop  comprehensive  M&S  index/directory;  assess  development 
initiatives;  employ  standardized  data  in  model  development;  and  continue  to  evolve  legacy 
systems.  Currently,  data  collection  is  underway  for  the  M&S  index  but  data  administration  is  in 
its  infancy  and  they  will  work  clos  ly  with  the  DoD  community  and  pursue  development  of  data 
standards  where  none  exist. 

They  have  the  charter  to  oversee  the  joint  framework  for  the  next  generation  M&S  environment 
for  constructive  simulations  with  the  initial  focus  of  replacing  the  ALSP  confederation  of 
models.  The  initiative  is  to  eliminate:  duplication,  interface  difficulties,  data  sharing  problems 
and  over  investment  of  critical  M&S  dollars. 

Beliefs  about  the  future:  simulation  can  lead  directly  to  improved  readiness  for  Aerospace 
forces;  simulation  provides  the  potential  for  significant  improvement  to  the  acquisition  process; 
and  simulation’s  promise  will  be  lost  without  warfighter  involvement  and  direction. 

Mr.  Dean  Free  and  LCDR  George  Flax:  Update  from  Navy  M&S  Office 
Published  in  the  “National  Defense  Authorization  Act  for  FY94”  on  July  27, 1993  by  the  SASC: 
The  committee  notes  the  continuing  lack  of  a  central  focus  in  the  Navy  on  M&S.  The  Navy  has 
been  in  the  forefront  of  M&S  at  the  technical  level,  though  most  of  the  activities  are  undertaken 
in  the  field  and  there  has  been  little  coordination  and  poor  oversight  over  these  activities.  A 
coordinated  focus  in  the  DON  could  substantially  improve  the  Navy’s  long  range  resources  and 
operational  planning. 

Current  view  is  that  there  is:  an  uncoordinated  use  of  models;  disparate  and  duplicative  model 
development  and  database  development  and  maintenance;  few  M&S  have  any  level  of  V&V; 
there  is  expensive,  ineffectual  participation  in  distributed  simulation  exercises  and 
demonstrations;  poor  transition  of  M&S  R&D  efforts  to  Navy  programs;  and  uncoordinated 
efforts  to  build  tools  for  the  future. 

The  proposed  management  structure  (shown  in  briefing)  is  for  an  M&S  Advisory  Council  (.with 
USMC  EA  and  USN  EA)  and  members  from  Acquisition,  Doctrine,  Training,  Operations, 
Marines?,  Assessment,  T&E,  and  Logistics.  There  would  be  M&S  fdads  in  each  of  the  above 
mentioned  areas  to  provide  vision  for  use  of  M&S  tools  within  the  functional  area,  etc.  There 
would  be  M&S  policy,  coordination  and  technical  support  for:  VV&A  instruction  and 
coordination;  support  for  common  simulation  framework;  support  for  advanced  simulation 
technology;  and  writing  and  maintaining  master  plan,  investment  and  strategy,  coordination  with 
DMSO  and  Joint.  The  M&S  Technical  Support  Office  (SPAWAR  31  integrator)  would  interface 
with  industry,  program  managers,  ARPA/ONR/DMSO,  N096,  CNA  and  other  FFRDCs, 
university  labs,  and  Naval  and  other  Service  warfare  centers. 

Summary:  Navy  has  an  M&S  mission;  organization  has  been  approved  by  the  Undersecretary 
and  includes  data  base  development  and  maintenance;  technically,  among  other  things,  NSS 
establishes  a  common  framework,  standards  and  protocols  and  is  a  prototype  for  JSIMS; 
distributed  experiments  are  a  key  part  of  applications;  there  will  be  an  automated  repository  for 
M&S;  and  VV&A  will  be  supported. 
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LCDR  George  Flax  is  the  Navy  representative  to  the  I/DBTWG  and  is  involved  with  the  ARMS 
program.  The  Navy  ARMS  system  will  have  over  350  sources  of  instance  data  and  they  are 
trying  to  get  the  instance  data  out  to  users  in  spite  of  having  no  standards.  They  are  providing 
the  user  with  multiply  sourced  data  and  trying  to  develop  tools  for  deconflietion  (a  semi- 
automated  process  using  mediators).  ARMS  is  a  centralized  repository  drawing  data  from  many 
sources  and  putting  it  into  a  common  format.  An  objective  is  to  consolidate  redundant  data 
gathering  and  distributed  efforts.  ARMS  will  ensure  reliable  sources  of  authoritative  data  use  in 
assessment,  be  an  electronic  clearing  house  for  data,  and  provide  blue  force  data  from  the 
NWTDB.  ARMS  is  secret  high  and  there  is  an  issue  on  how  widely  they  can  disseminate 
sensitive  data  (such  as  acoustic  signatures).  There  is  an  issue  with  authoritative  data  sources  in 
questioning  the  need  to  identify  some  sources.  ARMS  is  implemented  on  the  MAC  and 
currently  has  characteristics,  performance  and  effectiveness  data  and  a  partial  set  of  red  data.  It 
is  integrated  with  the  ITEM  Navy  system,  with  MARS  and  an  integrated  analysis  Navy  system. 

3.8  REPORTS  FROM  FUNCTIONAL  WORKING  GROUPS 
CDR  Mike  Lilienthal:  DoD  M&S  Master  Plan  Process 

CDR  Lilienthal  began  by  showing  a  chart  of  the  relationships  from  the  USD(A&T)  at  the  top  to 
the  Technology  Wording  Groups  (at  the  bottom)  to  which  I/DBTWG  belongs.  (The  I/DB  Task 
Group  is  now  known  us  the  Information  and  Databases  Technology  Working  Group 
(I/DBTWG).) 

The  DMSO  four  major  objectives  and  subareas  needing  objective  action  plans  are: 

(1)  Provide  Technical  Framework  for  M&S:  develop  simulation  architecture,  protocols  and 
standards,  and  data  base  and  repositories;  and  provide  common  tools  (VV&A)  and 
enhanced  communications. 

(2)  Develop  Authoritative  Representations:  for  environment,  terrain,  behavior,  and  systems. 

(3)  Integrate  Live-Virtual-Constructive  Simulations 

(4)  Broaden  M&S  Applications:  logistics,  mission  planning/mission  rehearsal,  acquisition 

The  overall  process  will  be  to:  identify  user  needs  through  the  Functional  Working  Groups, 
conduct  technical  assessments  (every  year)  through  the  Technology  Working  Groups,  establish 
Integrated  Process  Teams  (IPTs),  develop  investment  strategy  and  follow-through. 

The  tasks  of  the  TWGs  are  to  perform  technology  assessment,  project  guidance,  project 
mentoring  (making  sure  DMSO  projects  are  moving  along  on  the  right  path),  community 
coordination  (I/DB  meetings  and  TWGs  are  an  example),  and  participation  in  integrated  Process 
Teams.  The  existing  TWGs  are  Architecture,  Information  and  DataBase,  Verification, 

Validation  and  Certification,  and  Networking.  The  new  TWGs  will  be:  Environmental 
Representation,  Computer  Generated  Forces,  Human  System  Interfaces,  and  Interoperability  with 
C3I  Systems. 

The  tasks  of  the  FWGs  are  to  validate  needs,  prioritize  the  sub-objectives,  identify  major 
Component  programs  and  participate  in  Integrated  Process  Teams.  The  FWGs  are:  Education, 
Training  and  Military  Operations,  Analysis,  Research  and  Development,  Production  and 
Logistics,  and  Test  and  Evaluation. 

The  outline  for  the  Master  Plan  has  sections  on  introduction,  vision,  goal  and  objectives, 
organizations  supporting  Defense  M&S,  implementation  strategy  and  Appendices  for  each  of  the 
four  objectives  and  subareas. 


-29- 


DDR&E  has  identified  20  science  and  technology  (S&T)  areas,  one  of  which  is  M&S.  Each 
S&T  area  is  preparing  a  Technical  Area  Plan  (TAP).  The  team  preparing  the  M&S  TAP  is 
chaired  by  Dr.  Anita  Jones  and  members  are  from  the  EXCIMS  and  MSWG. 

The  FY95  DMSO  investment  plan  is  to  support  efforts  in  three  areas:  (1)  infrastructure  (where 
the  I/DBTWG  falls),  (2)  FY94 IPL  tails,  and  (3)  focused  areas  that  build  on  interoperability  and 
jointness  including  support  for  mission  rehearsal/mission  planning,  terrain,  CGF/SAFOR  (under 
consideration  are  logistics,  acquisition,  and  test  and  evaluation). 

The  key  organizations  supporting  DMSO  include:  M&S  ATD,  DISA,  ARPA,  ARPA/DISA 
AITS  JPO,  industry  and  academia,  Congress,  Intelligence  Community,  Services,  Combatant 
Commands  and  Joint  Staff. 

The  Senior  Readiness  Oversight  Council  (SROC)  meets  monthly.  One  of  the  DMSO  goals  is  to 
improve  readiness  through  simulations  by  applying  existing  simulation  technologies  to  improve 
readiness  training  and  monitoring  with  emphasis  on  Joint  readiness,  mission  planning  and 
rehearsal,  Reserve  Component  readiness. 

Dr.  Pat  Sanders  (presented  by  Mr.  Jim  Heusmann):  Report  from  the  Analysis  Functional 
Working  Group 

Analysis  supports  the  decisionmaker  in  the:  Requirements  Process  (JROC,  operational 
commanders,  CINCs,  etc.)  in  mission  area  analysis,  wargame  analysis,  battle  lab  output,  etc.;  the 
PPBS  Process  (DRB,  Programmers)  in  resource  allocation  studies,  value  added  analysis,  etc.;  and 
the  Acquisition  Process  (DAB,  PEO/PMs)  in  COEA,  affordability  assessments,  trade  studies,  etc. 

Effective  analysis  is  relevant,  responsive,  and  credible.  It  needs  M&S  that  address  the  questions 
of  interest  to  the  decisionmaker  and  include  factors  to  which  the  decision  is  sensitive;  accurate 
and  accessible  data  for  terrain,  atmosphere,  behavior,  weapons  performance,  cost,  etc.;  rapid 
scenario  generation;  analyst  friendly  M&S  tools;  PC-based  M&S;  confidence  in  analytic  tools 
(i.c.,  standards  and  VV&A  procedures  and  technology);  and  confidence  in  the  analyst. 

Mr.  Fred  Myers:  Report  from  the  Production  and  Logistics  Functional  Working  Group 

The  P&L  FWG  was  established  in  December  1991  by  ASD(P&L).  Its  charter  includes:  foster  a 
realistic  portrayal  of  production  (including  industrial  base  capabilities)  and  logistics  in  war 
games  and  simulation;  promote  use  of  real  world  models  to  bring  battlefield  operations  and 
maintenance  requirements  to  the  product  design  process;  and  to  create  a  methodology  for 
evaluating  P&L  i  vl&S  needs. 

They  surveyed  M&S  catalogs  against  their  needs  and  looked  at  over  200  M&S. 

They  identified  six  key  production  needs: 

(1)  Production/manufacturing  tools  for  integrated  product  and  process  development 

(2)  Technical  processes  and  data  models:  production  control  and  shop  floor  control  models 

(3)  M&S  support  of  remanufacturing  and  repair 

(4)  Coordination  with  national  and  international  standards  efforts  (need  common  datasets  and 
standards) 

(5)  Policy  and  management  direction  on  standardization  of  M&S 

(6)  Industrial  base  reconstitution 

They  identified  nine  key  logistics  needs: 

(1)  Higher  fidelity  representation  of  logistics  in  combat  models 

(2)  Credible  logistics  databases  and  data  collection  capabilities 

(3)  Planning/execution  tool  to  support  the  CINCs  in  OPLAN  assessments 
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(4)  Analysis  tools  to  study  effects  of  force  sizing  and  unit  realignment  on  logistics  infrastructure 

(5)  Acquisition  logistics  modeling 

(6)  Capability  to  quantify  implications  of  alternative  materiel  management  policy 

(7)  Analysis  capability  to  evaluate  ND1/COTS  equipment  performance  prior  to  purchase 

(8)  Interfaces  between  live,  virtual,  and  constructive  models 

(9)  Tools  to  support  logistics  considerations  in  the  PPBS  process 

Data  management  needs  are  for:  common  datasets  to  enable  sharing;  and  data  structure  metadata 
(to  make  it  easier  to  determine  data  usability,  independently  validate  available  data,  develop 
inter-related  models,  and  support  consistency  of  results  between  models). 

They  cannot  effectively  use  M&S  capabilities  without  common  data  sets  and  recognized  data 
structures — they  are  the  key  to  successful  M&S  implementation.  We  need  to  be  sure  the  key 
players  are  involved  in  the  data  activities  and  plan  for  evolving  systems  and  standards  since 
changes  are  inevitable. 

3.9  REPORTS  FROM  M&S  DATA  RELATED  PROJECTS 

Dr.  Henry  Heckathom:  Report  on  Environmental  Effects  in  Distributed  Interactive 
Simulation  (E2DIS)  Project:  An  Object-Oriented  Technology  for  Integrating 
Environmental  Effects  and  Distributed  Simulations 

Dr.  Heckathom  is  the  E2DIS  PM  and  Chairman  of  the  E2DIS  Technical  Management  Council. 
The  E2DIS  mission  is:  to  the  extent  that  they  impact  weapon  system  performance  and  attrition, 
provide  the  means  to  incorporate  sufficient  and  realistic  environmental  representations,  effects 
and  processes  consistently  in  DIS.  The  goals  are  to  provide  E&E2  infrastructure  for  sensor 
response  (recon,  surveil,  acquire,  track,  assess,  etc.),  platform  motion  (performance, 
trafficability,  velocity,  acceleration,  etc.),  and  use  of  environmental  models  in  decision  aids  and 
human  factors.  An  important  specific  goal  is  to  achieve  the  high  fidelity  simulation  of  sensor 
detection  of  targets. 

Modeling  issues  for  DIS  include:  adequate  modeling  of  E&E2  is  critical  to  realistic  simulation 
of  battlefield;  attrition  is  what  counts  (if  E&E2  doesn’t  effect  the  outcome  of  a  simulated  battle, 
its  inclusion  in  DIS  is  hard  to  justify);  sufficient-fidelity,  physics-based  models  and  data  are 
required  to  handle  the  variety  and  dynamics  of  the  real  world;  a  broad  range  of  scales  and  levels 
of  fidelity  are  needed;  DIS  must  handle  dynamic  environments  in  interactive,  real  -time  mode  for 
virtual  simulation;  and  consistent  representation  and  treatment  of  E&E2  is  critical  for  a  “fair 
fight." 

Fundamental  E2DIS  jobs:  methods  to  ensure  common  E&E2  across  DIS  cells;  finding  optimal 
solutions  by  testing  against  an  objective  function;  exploring  various  methods  to  achieve  goal  of 
high  correlation;  judging  solutions  by  how  well  they  measure  up  to  the  objective  function  but 
temper  with  pragmatic  requirements;  and  adopt  workable  development  process. 

The  E&E2  includes:  atmospheric  sciences  such  as  meteorology  and  its  effects  such  as  rain, 
snow,  etc.;  wind  flutter  effects  on  sensors;  they  don’t  do  emissions  but  do  obscurants  such  as 
smoke;  and  terrain.  E&E2  changes  include  the  effect  of  the  fog  rolling  in  on  the  way  a  sensor 
works.  They  may  need  to  work  in  four  dimensions:  X,  Y,  Z,  and  time. 

The  eight  tasks  are:  management  and  integration;  survey  requirements  and  capabilities; 
standards  (define  standard  database  structures,  transfer  formats  and  messages  to  allow  E&E2  to 
be  used  in  DIS);  environmental  representations  (develop  automated  methodologies  and 
procedures  to  put  numerical  environmental  data  into  standard  form);  environmental  effects  and 
processes  (provide  sufficient  fidelity,  etc.);  prototyping  and  experimentation  (prove  viability); 
and  constructive  simulations. 
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The  E&E2  DIS  4D  data  must  consist  of  basic  measurable  parameters;  must  maintain  temporal 
continuity;  consists  of  land,  sea,  air  and  space  data  that  is  physically  consistent  across  boundaries 
of  these  domains;  must  be  scalable,  must  be  VV&Ced;  and  must  be  object-oriented.  Currently, 
E2DIS  requires  two  distinct  databases:  atmosphere  and  terrain. 

The  basic  principle  of  OO  database  design  is  the  layered  interface  that  allows  the  user  to  specify 
input  and  output  to  a  database  without  concern  for  implementation  and  allows  different  users  to 
have  different  views  of  the  same  data.  They  plan  to  use  COTS  OODBMS  technology.  Contact 
Judith  Herbst  (STX  in  Washington)  with  respect  to  how  OODBMS  allows  user  to  specify  his 
data  requirement  and  hide  design,  whereas  RDBMS  does  not. 

On  data  standardization:  they  are  coordinating  with  existing  DoD  efforts  and  will  submit 
standards  to  the  CIM  process.  They  require  environmental  PDU  tables  and  there  are  a  set  of 
standard  transfer  formats  they  are  developing  or  considering  adopting  (see  briefing  chans). 

Question  about  use  of  BMDO  Analytic  Tool  Box:  it  provides  a  confidence  assessment  without 
true  VV&A. 

Dr.  John  Harding:  Report  on  Master  Environmental  Library 

This  is  a  DMSO  FY94  new  start  that  addresses  the  M&S  infrastructure.  The  rationale  for  this 
project  is  that  no  standard,  high  resolution  databases  exist  for  realistic  ocean,  atmosphere  and 
near  space  environment  data.  The  Services,  NASA  and  NOAA  share  access  to  large  sets  of 
environmental  observations  and  models  but  no  standard  extraction  methodology  nor  DoD  library 
applicable  to  M&S  exist.  Long  term  vision  is  of  an  environmental  library  having  features  of: 
general  M&S  applicability,  multi-service,  digital,  consistent  from  R&D  through  operations;  and 
containing  historical,  statistical,  and  4-D  data. 

Short  term  MEL  approach  is  to  build  a  rapid  prototype;  long  term  MEL  approach  is  to 
recommend  architecture  and  contents.  The  specific  tasks  are:  environmental  requirements; 
architecture;  climatological  and  fixed  databases;  integrated  synthetic  scenarios;  prototype 
demonstration;  prototype  evaluation;  and  management  and  integration. 

The  architecture  task  is  looking  at  browsing  techniques  such  as  WWW  Mosaic  and  the  NEONS 
browser  of  which  they  gave  a  very  good  viewgraph  demonstration.  The  Integrated  Synthetic 
Scenarios  task  is  identifying  relevant  4-D  scenarios,  selecting  and  acquiring  data  and  models  to 
create  prototype  scenarios,  creating  integrated  environmental  scenarios,  and  populating  the  MEL 
prototype  with  them. 

They  eventually  plan  to  work  with  E2DIS.  MELS  will  bring  data  from  NASA,  NOAA,  and  DoD 
together  which  may  require  dcconflicting  the  data. 

Mr.  James  Hammond:  Report  on  Naval  Battle  Force  Tactical  Trainer  (BFTT) 

BFTT  concept  is  that  the  ship  presents  the  most  effective  training  site  for  appropriate,  operational 
and  functional  training.  This  allows  ships  to  train  using  their  own  equipment,  system 
configurations,  and  operational/casualty  procedures.  Enhanced  training  efficiency  will  result  as 
training  redundancy  is  identified/eliminated,  a  necessary  reality  in  terms  of  future  down-sizing  of 
the  Navy. 

BFTT  focus:  incremental  development  by  baselines;  leverage  off  of  existing  training  systems; 
utilize  Navy  standard  databases;  coordinate  and  integrate  with  other  programs;  build  only  what  is 
necessary;  use  existing  contracts  when  possible;  and  accelerate  to  support  ATO/TTS  and 
downsizing.  BFTT  will  support  operator/unit/team  training,  Afloat  Training  Organization 
(ATO)/Tactical  Training  Strategy  (TTS);  Battle  Group/Battle  Fleet  commander  training;  and 
Joint  training. 
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The  BFTT  gameboard  is  4000  X  4000  miles,  altitude  300K  FT,  ocean  depth  16K  FT,  worldwide, 
and  2600  DIS  entities  (2K  guided  missiles,  500  countermeasures,  and  100  environments). 
Developmental  test  IIA  October  94:  multi-ship/shore  connectivity,  battleforce  and  unit-level 
training,  multi-warfare  (AAW,  ASW,  and  EW),  DIS  2.0.3,  technical  measures  are  entities  =  > 
than  100,  latency  <  300  MS,  synchrony  >  98%  and  debrief  15  min  (ship  level)  arid  90  min  (force 
level).  The  95  development  test  is  stepped  up  to  show  JMCIS  interface  and  technically  to  handle 
entities  =  >512,  latency  <  300MS,  synchrony  >  99.9%  and  same  debrief.  They  plan  on 
participating  in  future  I/ITSECs  and  STOW-Ex’s,  DT-IIIA  (FY96),  and  DT-IIIB  (FY97). 

The  BFTT  distributed  backbone  is  DIS,  and  it  uses  models  and  databases  to  implement  DIS.  It  is 
a  customer  of  validated  models  and  standard  databases  and  is  establishing  a  Database  WG. 

BF'TT  database  issues  include:  need  to  interface  with  most  shipboard  embedded  trainers  which 
don’t  work  with  NWTDB  data;  need  to  interface  with  many  shore  sites;  must  fill  legacy 
databases  under  control  of  others  (mostly  NWTDB);  must  be  interoperable  across  DoD;  must 
have  flexible  database  architecture  to  allow  for  future  growth  and  at  same  time  support  legacy 
databases;  DIS  architectures  achieve  fidelity  by  employing  real  data;  minimize  traffic  by  pre¬ 
positioning  correlated  databases;  following  DoD  data  modeling  approach;  choosing  optimum 
database  schema  (balance  between  OO  (future)  and  legacy  (RDBMS  DoD  installed  base);  and 
legacy  intelligence  databases  are  classified. 

SPAWAR/NAVAR  is  addressing  next  generation  computer  resources.  DISWG  is  tasked  to 
define  a  Navy  Standard  Database  by  1997:  working  with  standards  groups  and  industry  to  define 
potential  COTS  products;  ARL-UT  Research  heads  DB  architecture  WG;  recommend  NAVSEA 
use  forum  to  develop  candidate  database  development  across  combat  system  products. 


-33- 


4.  REPOSITORY  DISCUSSION  ON  MONDAY  JULY  11, 1994  AT  1700  AT  L'MSO 

This  meeting  was  held  to  discuss  and  agree  on  the  MITRE  task  objectives  and  coordinate  that 
effort  with  the  Data  Standards  TF  Repository  Subgroup  effort  co-chaired  by  Jim  Augins  and  Pete 
Valentine. 

Jim  Augins  said  that  the  repository  design  should  be  driven  by  the  life  cycle  maintenance  of  the 
data.  Attention  needs  to  be  paid  to  mechanisms  for  use  and  maintenance  of  the  repository  as  well 
as  the  telecommunications  aspects  of  federated  systems  (for  distributed  repositories). 

Agreement  to  MITRE’s  task  deliverables: 

MITRE’s  approach  is  to  define  the  repository  for  DoD  M&S  military  applications  in  the  five 
DMSO  functional  areas  in  support  of  the  warfighter 

DESCRIPTION  PRODUCTS 


1.  Mission  Statement  Activities  and  product 
requirements 

2.  Standards  affecting  #1 
Subset  of  #1 

Identification/issues/impact/  and  the 
application  to  M&S 

3.  CONOPS  for  each  role  in  order  to  serve 
needs  identified  in  #1.  Roles  are:  FDAd, 
study  director,  data  center,  VV&C  system 
developers,  DBAd,  Services’  data 
acquisition  (deconfliction) 


IDEF1X  data  model  of  repository 

Issues:  list/report  for  current  use  and  for 
3-5  years 

IDEFO 


Repository  Subgroup  Tasks: 

1.  Organize  itself 

2.  Work  with  MITRE  as  subject  experts  on  #1  and  #3 

3.  Review  #2 

4.  Assess,  prioritize,  and  recommend  at  the  end  of  the  MITRE  effort 
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5.  COMPLEX  DATA  TASK  FORCE  MEETING  NOTES 
5.1  AGENDA 

WEDNESDAY  JULY  13,  1994,0800-  1700 

0830-0900  Review  of  goals,  strategy,  accomplishments:  Ms.  Iris  Kameny 
0900-1000  Object  Oriented  Modeling  Techniques:  Ms.  Elaine  Ward 
1000-1030  Break 

1030-1230  C2  Core  Model:  Dr.  Robert  Walker 
1230-1330  Lunch 

1330-1400  Data  Modeling  Needs  for  the  National  Ground  Intelligence  Center: 
Ms.  Janet  Morrow 

1400-1430  Discussion  of  Three  Taxonomies:  Iris  Kameny 
1430-1500  Break 

1500-1630  Complex  Data  Categorization  Discussion:  Mr.  Len  Seligman  and 
Mr.  Peter  Valentine 

1630  --  1700  Wrapup:  accomplishments,  next  meeting  and  goals,  etc. 


5.2  ATTENDEE  LIST 
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5.3  COMPLEX  DATA  ISSUE  DISCUSSION 

Ms.  Iris  Kameny;  Review  of  Goals,  Strategy,  Accomplishments 

Iris  Kaineny  began  by  reviewing:  the  organization  of  the  Complex  Data  Task  Force,  previous 
activities,  and  long  term  and  short  term  goals.  The  long  term  goals  are  to  develop  an  M&S 
guideline  to  data  modeling  and  standardization  of  complex  data  types  in  coordination  with  CIM. 
Near  term  goals  are:  (1)  definition  and  categorization  of  complex  data;  (2)  developing  complex 
data  models  and  standards  (as  well  as  data  modeling  techniques  and  standards)  from  pilot  studies 
by  participating  projects;  and  (3)  coordination  with  CIM  as  to  issues  and  problems  and  suggested 
extensions  to  data  modeling  and  standards. 

The  definition  and  non-exhaustive  categories  of  complex  data  agreed  to  in  the  April  6-7,  1994 
meeting  can  be  found  in  Section  3.4,  “  Results  from  I/DB  Task  Forces  and  Subgroups”,  under 
Mr.  Len  Seligman:  Report  from  the  M&S  Complex  Data  Task  Force  Subgroup  on 
Categorization. 

Items  of  interest  that  occurred  between  the  April  meeting  and  this  July  meeting  include:  ( 1)  the 
DMA  MC&G  standardization  effort  funded  by  CIM  is  underway;  (2)  the  DIRS  model  definitions 
paper  has  been  delayed;  and  (3)  the  Workshop  on  Data  Representations  in  Scientific 
Computation  hosted  by  the  Climate  System  Modeling  Group  at  LLNL  will  be  held  at  Villa 
Tassajara,  Pleasanton  Calif.,  August  16,1994  (they  are  addressing  things  such  as  grid 
representations,  geometry,  units  and  a  host  of  parameters  and  attributes  currently  incorporated 
into  scientific  DBMSs  and  code  development  environments).  Duane  Hufford  provided  an 
“example  metadata  model  for  mapping  domain  collisions  due  to  data  element  associations”  (that 
is  included  in  the  Appendix). 

Issues/briefs  that  the  group  wanted  to  cover  during  this  meeting  included:  (1)  results  of  the  first 
IEEE  Metadata  Workshop  (2)  progress  toward  development  of  taxonomies  (3)  brief  and 
discussion  of  object-oriented  data  and  OO  data  modeling;  (4)  discussion  and  evaluation  of  C2 
Core  Model  (5)  brief  from  Janet  Morrow  on  National  Ground  Intelligence  Center 

Ms.  Elaine  Ward:  Object  Modeling:  A  Solution  to  Complex  Data  Challenges 
Most  complex  data  challenges  are  a  result  of  organizational  dynamics  and  technological 
advances  and  will  not  be  solved  by  object  modeling  or  any  other  modeling  technique. 
Organizational  dynamics  affect  inter-and  intra-  organizational  data:  business  process  (domain) 
changes;  instance  data  changes;  mappings  among  different  organizational  and  systems  models; 
data  dependencies  or  business  rules;  data  semantics  inconsistencies  among  data  models;  and  lack 
of  normalization  in  legacy  systems.  Technological  advancements  affect  issues  relating  to  data 
integrity  and  synchronization:  heterogeneous,  distributed  environments;  data  interoperability; 
data  sharing;  and  user  access  privileges/ownership. 

Traditional  data  modeling  techniques  are  static  and  associate  primitive  types  (entities)  with 
attributes  without  capturing  semantics.  A  problem  is  that  instances  that  do  not  relate  to  each 
other  may  be  compared  by  accident  (e.g.,  one  may  try  to  select  or  join  across  data  fields  that  are 
the  same  syntactic  data  type  (e.g.,  character)  but  are  not  related,  such  as  name  of  country  and 
name  of  organization).  Relationships  of  complex  data  include  inheritance,  composition 
(hierarchies,  complex  structures,  BLOBS). 

Key  aspects  of  object  orientation  are  not  really  agreed  to  yet  but  there  is  general  agreement  on 
the  following  basic  concepts  for  OO.  Static  characteristics:  unique  identity;  encapsulation 
(enforced  association  of  attributes  and  behavior);  information  hiding;  class  concept  (template 
that  represents  common  instances);  and  complex  relationships  (e.g.,  inheritance).  Dynamic 
characteristics  are:  notion  of  state;  message  passing;  well-defined  interface  (access  through 
methods);  and  polymorphism  (ability  of  two  or  more  classes  to  respond  to  the  same  message 
appropriately). 
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Important  comparisons  between  data  and  object  models:  both  models  promote  reuse  and 
synchronization,  promote  data  sharing  and  avoid  stovepipe  systems.  There  is  a  difference 
between  the  models  in  focus/scope  in  that  top-level  object  models  have  a  domain  focus  problem 
(often  including  people  and  other  resources),  and  conceptual  data  models  tend  to  focus  only  on 
the  data  to  be  stored  in  the  database.  There  are  differences  in  content:  object  models  include 
problem  domain  classes,  their  behaviors  and  relationships  among  the  objects  while  data  models 
include  mainly  entities,  attributes  and  basic  relationships  (but  sometimes  include  a  mechanism 
for  strong  typing).  She  showed  an  example  of  entity  vs  class  for  a  tank  where  the  tank  class 
included  an  identifier  and  state  that  was  not  present  in  the  tank  entity.  She  also  showed  how 
objects  could  inherit  polymorphic  attributes  through  value  restrictions  on  information  types  in 
their  subsets.  00  also  enables  the  modeling  of  the  behavior  of  data  because  it  can  address  both 
static  and  dynamic  characteristics. 

Issues:  objects  are  in  the  eye  of  the  beholder  (objects  are  abstractions  that  can  be  used  to  model 
various  views,  or  represent  large-or  fine-grained  information)  and  underlying  object  models  used 
by  different  technologies  are  not  necessarily  compatible.  There  are  different  types  of  objects 
(e.g.,  corporate  level  objects,  subject  area  domain  objects,  system  architecture  objects,  analysis 
and  logical  design  objects,  physical  design  objects,  implementation  objects)  and  related  models 
developed  with  different  OO  techniques  do  not  easily  man  to  each  other  nor  do  models 
developed  with  OO  and  non-00  techniques.  The  briefing  goes  into  some  detail  on  the  problems 
In  relating  models  in  each  of  these  areas. 

Summary:  (1)  mixing  and  matching  of  OO  analysis  and  OO  design  methods  takes  up  front  work 
and  is  not  always  seamless;  (2)  one  needs  to  establish  a  well-defined  process  before  beginning 
data  modeling  activities;  (3)  pilot  studies  help  in  identifying  potential  approaches;  (4)  object 
models  can  provide  significant  enhancements  over  traditional  models;  (5)  to  reap  benefits  from 
OO  one  must  use  proper  tools,  and  ensure  that  tools  and  methods  can  integrate  with  process  and 
implementation  technologies. 

Object  oriented  issue  discussion  included:  ( 1)  concern  in  confusing  class  and  type;  (2)  Elaine 
Ward  mentioned  a  comparison  document  of  00  methodologies  that  was  available  from  MITRE; 
and  (3)  the  CCTT  domain  analysis  and  DISA/CIM  design  analysis  may  have  relevance  here. 

Dr.  Robert  Walker:  C2  Core  Model 

The  purpose  of  data  models  is  to  provide  a  high-level  specification  of  information  inputs  and 
outputs  of  functional  processes,  particularly  those  information  items  subject  to  exchange.  A  data 
model  provides  a  consistent  basis  for  data  element  standardization.  The  C2  Core  Model  began  as 
the  NATO  C2  Generic  Hub  data  model.  The  concept  was  to  define  a  core  C2  data  model  that 
could  be  the  common  starting  point  or  hub  for  data  models  in  all  C2  subfunctional  areas.  Each 
subfunctional  area  would  be  an  extension  to  the  generic  hub. 

Dr.  Walker  briefed  during  the  2-day  I/DBTWG  meeting,  but  agreed  to  come  to  this  TF  meeting 
and  go  into  more  detail  about  the  C2  Core  Model.  IDA  has  proven  that  data  models  can  be 
developed  at  the  core  level  and  extended.  The  question  being  asked  by  our  M&S  community,  is 
whether  we  should  start  with  the  C2  Core  Model  and  build  our  own  M&S  extension(s). 

The  C2  Core  Model  is  missing  references  to  certain  other  models  in  the  functional  areas  of 
personnel,  logistics,  finance  and  accounting. 

Last  year  the  MCEB  directed  the  C2  FDAd  to  build  the  C2  Core  Model  from  the  NATO  Generic 
Hub  model  so  as  to  be  integrable  with  the  DoD  Enterprise  Data  Model.  This  required  excluding 
the  object-item  and  object-type  concepts  from  the  C2  Core  Model.  The  results  of  this  exclusion 
has  been  to  explode  the  C2  Core  Model.  In  data  modeling,  there  is  often  a  tradeoff  of  precision 
vs  complexity.  The  current  US  position  is  to  have  precision  and  complexity. 
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Location  and  time  are  complex  concepts  with  many  representations  that  need  to  be  modeled  in 
the  data  model. 

Location  in  the  DoD  Enterprise  model  is  separate  from  the  DMA  geo-loc  feature  for  C2.  There 
are  hundreds  of  ways  to  represent  location.  The  C2  Core  Model  uses  an  abstract  data  type  called 
location  with  thirty-five  different  representations.  The  multiple  representations  need  to  be 
formalized.  An  example  was  (1)  establish  the  DMA  standard  as  the  DoD  standard,  (2)  establish 
a  UTM  “standard”,  (3)  have  standard  algorithm  for  translating  the  standard  to  UTM,  and  (4)  have 
standard  algorithm  for  translating  UTM  to  the  standard. 

Location  would  only  be  connected  to  object  items.  An  associate  entity  would  be  used  to  indicate 
how  location  is  calculated  for  an  aggregate  object  (e.g.,  location  of  unit  could  be  a  geometric 
weighting,  location  of  commander,  etc.).  Accuracy  would  be  an  attribute  of  the  associative  entity 
while  precision  would  be  an  attribute  of  location.  For  each  material  and  location,  there  would 
need  to  be  a  key  to  the  different  instances  of  the  location  code  (to  translate  to  different 
representations). 

For  signals  of  an  unknown  object,  “signal”  would  be  a  feature  of  the  object.  One  could  keep 
track  in  the  “material”  entity  of  position  and  speed  or  could  have  position  and  speed  as  part  of 
location. 

He  discussed  migration  of  existing  systems  to  the  C2  Core  Model.  An  example  is  JOPES.  One 
way  is  to  include  the  whole  JOPES  model  as  is  and  enclose  it  in  dotted  lines  (as  a  black  box). 
Another  is  to  define  new  independent  entities  in  JOPES  and  integrate  those  with  the  C2  Core 
Model  entities. 

He  discussed  logical  modeling  of  point,  line,  area,  polygon.  There  is  only  one  attribute  with 
coordinates  in  the  data  model  and  that  is  a  point.  An  associate  entity  is  used  to  model  a  line  in 
terms  of  two  points;  areas  and  polygons  are  handled  similarly.  An  exception  is  a  perforated  area, 
i.e.,  an  area  with  holes  or  something  missing. 

He  said  that  we  could  replace  “location”  with  other  concepts  needed  by  the  M&S  community  for 
our  data  model. 

He  is  very  interested  in  having  the  M&S  community  and  the  intelligence  community  try  to  use 
the  C2  Core  Model  and  point  out  problems,  lacks,  etc.  He  would  like  us  to  think  of  this  as  a  set 
of  specifications  to  complete  for  DoD. 

In  the  Fire  Support  extension  to  the  C2  Core  Model,  there  is  a  concept  of  object  type 
effectiveness  (e.g.,  material  type  and  material  type  (tank  on  tank)  has  a  data  element  Pk  that  has 
values).  The  value  is  modeled  against  the  two  dimensions.  A  target  is  the  focus  of  an  action 
against  a  thing  that  actually  exists.  A  target  doesn’t  become  a  thing  until  it  is  part  of  an  action. 
We  need  to  find  out  more  about  this  way  of  modeling  weapon  performance  data  elements. 

The  C2  Core  Model  is  the  source  of  all  C2  data  elements  but  not  of  their  implementation.  The 
DoD  Data  Model  consists  of  entities  and  attributes  that  have  gone  through  the  approval  process. 
The  DoD  Enterprise  Data  Model  has  not  gone  through  the  approval  process.  JOPES  +  the  C2 
Core  Model  have  been  circulated  for  comment.  The  M&S  community  goal  is  to  integrate  our 
data  models  with  the  DoD  Data  Model  in  accord  with  the  Enterprise  Data  Model,  using  the  C2 
Core  Model  as  guidance.  (He  said  this,  but  in  practical  terms  I  am  not  sure  what  “in  accord  with” 
or  “as  guidance”  means.)  The  C2  Core  Model  is  an  option  Walker  believes  we  won’t  get 
anywhere  without;  we  need  to  use  it  to  get  integration  about  data  M&S  needs  to  use  and  share 
with  the  operational  community.  We  need  a  strategy  to  get  data  requirements  into  the  DoD 
process.  We  need  to  find  resources  in  the  DoD  community  to  help  us  do  M&S  data  modeling. 
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He  said  for  us  to  pick  a  day  to  work  with  them  and  they  would  try  to  find  the  time.  The  C2  Core 
Model  document  is  a  high  level  document  to  use;  we  may  also  need  something  more  detailed. 

Ms.  Janet  Morrow:  National  Ground  Intelligence  Center 

Richard  Bernstein  has  been  working  with  Ms.  Morrow  on  intelligence  data  for  ground  systems. 
They  have  now  joined  with  the  tactics  and  doctrine  center.  They  deal  with  data  and  information 
at  all  levels  of  security. 

They  distribute  their  data  on  CD-ROM  and  it  can  be  accessed  via  hypertext  links.  Their  database 
plan  calls  for  the  use  of  hypermedia  to  link  different  types  of  information  such  as  hypertext 
documents,  weapons  characteristics  in  an  Oracle  database  and  an  object  base.  They  have 
complex  intramodel  relationships.  They  have  the  ability  to  extract  data/information  from  free 
text  using  expert  system  tools.  The  multimedia  information  includes  combat  instruction  sets, 
video,  pictures,  and  graphs.  Their  C2  database  is  in  template  form  showing  unit,  equipment  and 
who  communicates  with  who. 

She  thinks  multiple  relationships,  aggregations,  metadata,  dynamic  data,  etc.  all  need  to  be 
focused  to  get  the  information  people  need.  If  this  were  done  correctly,  one  should  be  able  to 
create,  for  example,  a  generic  tank  object  to  play  in  DIS. 

Their  data  quality  is  usually  very  good  since  the  analyst  knows  the  area  well  and  may  use  CAD, 
engineering  models,  projections,  tests  at  Aberdeen  to  do  validation.  There  is  a  measure  of  the 
degree  of  confidence  in  the  quality  of  the  data.  There  are  four  confidence  levels:  1  is  certain,  4  is 
an  engineering  guess.  It  is  often  hard  to  capture  the  data  because  of  complexity,  the  data  is  not  in 
3NF,  and  the  same  data  may  be  in  many  different  categories. 

They  would  like  to  talk  with  us  mote  about  data  models  and  M&S. 

Jim  Aiigins:  Discussion 

Multimedia  data  is  complex  data.  He  has  information  on  multimedia  standards,  products,  and 
research.  This  includes  the  communication  standards  used  to  access  multimedia  and  hypertext 
information  over  the  internet.  He  will  be  publishing  a  Mosaic  page  with  a  Universal  Resource 
Locator  (URL)  for  the  repository  subgroup.  The  Mosaic  page  will  include  a  bibliography  of  the 
information  he  has  in  his  notebooks  (multimedia,  exchange  standards,  etc.).  His  next  step  is  to 
send  out  a  set  of  abstracts  and  URLs  over  email  about  multimedia  and  communication  and 
exchange  standards. 

Discussion 

ODBC  is  an  Open  Data  Base  Connectivity  standard  agreed  to  by  a  consortium  of  DBMS 
vendors.  It  allows  access  to  different  DBMS  services. 

The  Microsoft  standard  uses  an  object  library  incorporated  into  the  application.  This  acts  as  an 
ODBC  DMS.  It  is  needed  to  solve  SQL  incompatibility  problems.  NIST  is  involved  in  these 
standards. 

5.4  SUBGROUP  ORGANIZATION 

The  TF  agreed  to  do  away  with  the  Complex  Data  subgroups  leaving  the  Complex  Data  Task 
Force  with  no  subgroups.  The  TF  goals  remain  the  same:  to  continue  to  work  on  data  modeling 
of  complex  data,  guidelines  to  modeling  and  standards  for  complex  data,  and  pilot  studies.  It 
was  agreed  to  make  the  Taxonomy  Subgroup  issues  and  goals  the  responsibility  of  the  Data 
Standards  Task  Force  Repository  Subgroup. 

Summary:  during  the  April  meeting  the  TF  agreed  on  a  definition  for  complex  data  and  defined  a 
non-exhaustive  list  of  categories  of  complex  data.  During  this  meeting,  it  was  suggested  that 
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future  pilot  studies  for  complex  data  be  selected  from  among  the  ongoing  M&S  projects,  e.g., 
CCTT,  UTSS,  ARMS,  CENTCOM’s  CFDB.  Another  possibility  suggested  at  an  earlier 
meeting,  is  to  select  Navy  and  Air  Force  weapons  performance  data  for  pilot  studies  as  a 
complement  to  the  Army  TRAC  pilot  study  and  as  a  way  to  look  at  data  modeling  and  standards 
across  a  large  part  of  the  DoD  weapons  performance  data. 

The  new  effort  getting  underway  for  a  Joint  Simulation  System  (JSIMS)  was  discussed.  JSIMS 
will  have  an  object  oriented  base.  There  is  a  need  to  understand  how  the  object-oriented 
approach  will  fulfill  the  DoD  8320  requirement  and  IDEF1X  data  modeling.  This  TF  needs  to 
interact  with  JSIMS  to  understand  their  intent  with  respect  to  data. 

5.5  FUTURE  MEETINGS 

There  was  no  future  meeting  scheduled. 
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6.  DATA  STANDARDS  TASK  FORCE  MEETING  NOTES 
6.1  AGENDA 

THURSDAY  JULY  14,  1994,  0800-  1200 
0800-08 1 5  Review  of  past  meetings  and  objectives 

08 1 5-0845  Report  on  Needs  for  DIS  Data  Standards  paper:  MAI  W alt  Swindell 

M&S  REPOSITORY  SUBGROUP  SESSION 

0845-0900  Repository  Goals  and  Strategy:  Mr.  Jim  Augins 

0900- 1000  M&S  Repository  Design  Discussion:  Mr.  Mike  Gorman 

1000-1030  NASA  Distributed  Active  Archive  Center  (DAAC):  Mr.  Ken  McDonald 

1030-1130  Repository  issues  and  next  steps:  Mr.  Jim  Augins 

1130-1200  Wrapup:  led  by  Iris  Kameny 


6,3  DATA  STANDARDS  ISSUE  DISCUSSION 


Ms.  Iris  Kameny:  Review  of  Past  Meetings  and  Objectives 

Iris  Kameny  opened  the  meeting  and  quickly  reviewed  the  purpose  and  goals  of  the  Data 
Standards  TF  (condensation  of  what  she  presented  on  July  1 1th  at  the  opening  session  of  the 
I/DBTWG,  her  briefing  charts  are  in  that  Appendix).  The  Data  Standards  Task  Force  has  formed 
two  Subgroups:  the  DIS  Data  Standards  Subgroup  chaired  by  Walt  Swindell  and  Army/TRAC 
ar.d  the  Repository  Subgroup  co-chaired  by  Jim  Augins  and  Pete  Valentine.  The  DIS  Data 
Standards  Subgroup  held  a  meeting  in  Orlando  to  organize  the  writing  of  a  DIS  paper  to  be 
presented  at  the  September  DIS  meeting  (the  paper  has  been  accepted.)  The  Repository 
Subgroup  held  a  short  meeting  during  the  previous  I/DBTWG  meeting  in  February  and  asked  to 
use  most  of  the  time  in  this  meeting  for  a  Repository  Subgroup  meeting.  The  first  hour  of  this 
meeting  was  devoted  to  the  Data  Standards  TF  and  the  last  3  hours  to  the  Repository  Subgroup. 

MAJ  Walt  Swindell:  DIS  Data  Standards  Subgroups 

Walt  Swindell  was  scheduled  next  to  speak  about  the  DIS  Data  Standards  Subgroup  activities 
which  resulted  in  the  paper  “DIS  Needs  for  DoD  Data  Standards"  co-authorcd  by  members  of  the 
Subgroup.  Walt  was  unexpectedly  called  back  to  Leavenworth  for  the  birth  of  his  fourth  son  and 
so  Iris  Kameny  presented  his  brief  on  the  DIS  paper.  Essentially  the  puipose  of  the  paper  is  to 
convince  the  DIS  community  that  the  DIS  PDUs  will  not  be  adequate,  efficient  or  cost  effective 
for  supporting  interoperability  among  models  and  simulations.  Instead,  data  standards  based  on 
the  DoD  data  modeling  and  standardization  process  are  recommended.  The  paper  discusses  the 
evolution  of  data  standards,  the  DIS  vision  and  the  lack  of  data  standards  in  that  vision. 
Advantages  of  using  data  standards:  cost  efficiency  in  reuse  of  data  and  data  standards,  increased 
credibility  of  DIS  exercise  results,  reuse  of  models,  interoperability  of  M&S  and  operational 
world  (given  operational  world  will  be  developing  and  using  data  standards),  and  affect  of  data 
standards  on  use  of  legacy  instance  databases.  The  paper  describes  the  standardization  process 
with  examples  of  IDEFO  and  IDEF1X  modeling.  The  paper  ends  with  a  vision  of  DIS  using  data 
standards  and  a  suggested  roadmap  as  to  what  DIS  should  do  in  the  future. 

Dr.  Jim  Hammond:  Security  Issue 

BFTTs  raised  a  security  issue  since  they  are  working  with  DIS  but  need  to  use  classified  data. 
This  issue  has  been  folded  into  the  overall  recommendation  to  DMSO  that  a  new  I/DBTWG  task 
force  be  formed  to  address  data  security  requirements  and  get  M&S  data  security  requirements  to 
DISA/JIEO/CISS  for  attention  and  solution. 

REPOSITORY  SUBGROUP  MEETING 

Mike  Gorman  reminded  us  that  the  review  of  the  MITRE  M&S  Mission  Statement  part  of  their 
repository  requirements  task  was  due  for  review.  On  7/29  they  plan  to  send  out  the  IDEF1X 
diagram  of  the  repository  objects  and  their  relationships  to  each  other.  He  asked  how  they 
should  send  this  non-textual  material,  Apple  BINHEX,  or?  The  group  decided  it  should  be  sent 
by  fax  and  Jim  Augins  took  responsibility  for  getting  everyone’s  fax  numbers  together. 

Mr.  Kenneth  McDonald:  EOSDIS  Information  Management 

Pat  Liggett  from  JPL,  who  is  supporting  NASA,  arranged  for  this  briefing  so  we  could  learn 
more  about  the  DAACs  as  repositories  and  benefit  from  their  lessons  learned.  Ken  is  at  the 
NASA  Goddard  Space  Flight  Center  and  can  be  contacted  email:  ken.mcdonald@eos.nasa.gov 

The  goals  of  EOSDIS  Version  0,  which  was  started  in  FY91  during  the  definition  of  the  EOSDIS 
Core  System  (ECS),  were:  to  prototype  uniform  search  and  order  access  to  data  held  at  Version  0 
DAACs,  provide  operational  capability  in  July  1994,  solicit  input  from  user  community  during 
design  and  operation  of  Version  0  elements,  and  gain  experience  that  could  be  applied  to  ECS. 
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When  they  started  up  each  archive  had  an  existing  system  and  they  needed  to  build  a  server  layer 
to  run  on  top  of  each  system.  They  needed  to  translate  everything  in  the  server  layer  to  interface 
with  the  client.  For  in-depth  information  about  datasets,  they  currently  use  local  interfaces  that 
are  unique  for  each  system.  They  also  needed  different  search  aids,  e.g.,  for  text,  for  geographic 
searches.  They  still  need  to  get  a  feel,  during  the  test  phase,  for  how  much  they  can  open  the 
system  up  to  users,  how  much  traffic  they  can  accommodate  on  individual  systems,  over  the 
network,  etc.  It  took  about  three  years  (1991-1994)  for  Version  0  to  become  operational.  The 
EOSDIS  Information  Management  System  (IMS)  VO  is  a  client/server  system. 

We  can  get  a  paper  on  VO  lessons  learned. 

Not  all  NASA  data  is  accessed  through  DAACs.  There  are  many  smaller  sites  at  universities  and 
labs  that  can  be  found  through  directories  and  then  accessed  directly.  For  access,  they  are  using 
the  Hierarchical  Data  Format  (HDF)  as  a  minimal  standard.  This  is  adequate  for  subsample 
images,  low  resolution  images,  and  images  designed  for  browsing  but  not  for  high  resolution 
images.  Mosaic  offers  an  advantage  in  browsing  and  they  have  also  looked  at  the  NEON 
browser  for  NEON  distributed  systems.  The  intelligence  image  systems  are  heading  in  the 
NEON  direction. 

The  briefing  charts  show  the  architecture.  There  is  a  global  changes  master  directory  that  has 
brief  descriptions,  etc.,  guide  information  using  Mosaic  hypertext  links  to  data  sets  etc.,  data  sets 
with  pointers  to  granules  (smallest  unit  of  data  independently  managed)  and  pointers  to  inventory 
metadata  which  contains  descriptions  of  granules  and  information  about  how  to  select  and  obtain 
a  subset,  and  then  a  browse  mechanism  for  browsing  granules  to  aid  in  selection.  The  IMS 
supports  directory  search,  inventory  search,  guide  search,  visual  aids  (graphic  displays,  retrieve 
and  display  precomputed  browse  products),  results  integration  (sort,  merge,  and  select),  and 
product  requests.  It  is  based  on  a  client/server  model  using  a  message  passing  approach.  There 
is  reuse  of  the  guide  software  which  is  text  search  and  hypertext  navigation  built  on  WAIS, 
WWW,  and  X-Mosaic.  The  servers  are  heterogeneous  HW/SW  using  heterogeneous  RDBMS 
while  the  clients  were  developed  on  multiple  Unix  workstations  but  currently  running  on  a  single 
SGI. 

Charts:  Included  in  the  Appendix  for  this  section  are  the  latest  versions  of  several  data 
models 

(1)  The  DMSO  Database  Directory  Fully  Attributed  Logical  Data  Model  (IDEF1X)  modified  by 
JDBE  8  July  1994 

(2)  The  DMSO  Model  and  Simulation  Directory  Fully  Attributed  Logical  Data  Model  (IDEF1X) 
modified  by  JDBE  8  July  1994 

(3)  DIS  Exercise  Data  Model:  authors  Walt  Swindell  (TRAC)  and  Peggy  Gravitz  (COLSA 
Corp.),  1  July  1994 

Mr.  Jim  Augins:  Presented  an  outline  for  the  repository  meeting 

(1)  get  together  a  list  of  issues 

(2)  prioritize  the  future  things  the  group  needs  to  do 

(3)  get  organized 

(4)  decide  if  this  is  the  right  forum  for  getting  together  or  do  we  need  to  create  another  forum 

(5)  give  Peter  a  chance  to  talk:  Peter’s  issue:  interoperability  of  modeling  information 

Issues 

(1)  Lack  of  standards  for  repositories 

—  state  why  FIPS  156  and  DDRS  are  insufficient; 
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—  Chicn  needs  to  provide  DDRS  specifications  to  JDBE  and  MITRE  in  order  for  them  to 
evaluate  the  DDRS 

—  August  19,  DDRS  steering  committee  meeting:  can  Chien  attend? 

(2)  Definition  of  repository  (different  kinds  of  repositories) 

(3)  There  are  multiple  requirements  for  M&S  repositories 

(4)  Multiple  M&S  repository  efforts  need  coordination 

(5)  Repository  needs  beyond  data  models:  MITRE  (e.g.,  data  centers,  M&S  reuse  repositories, 
etc.) 

(6)  Communication  standards  for  repositories 

—  PDU/DIS,  others,  internet,  classified/unclassified 

—  how  repositories  will  interoperate,  communication  standards,  networks,  system  load,  etc. 

—  DIS  needs  to  address  this  with  respect  to  network  distribution  of  databases  for  a  DIS 
exercise 

—  Resources  Information  Management  System  (RIMS),  Jim  Watson  will  send  Iris  this 
document 

(7)  Short  term  requirements 

—  M&S  FDAd  needs 

—  Data  repository  efforts 

(8)  Physical  data  model  as  an  end  result  (schema) 

(9)  Classification/security  releasability  issues 

—  Implementation  of  a  repository  as  influenced  by  these  issues 

( 10)  Need  for  multiple  repository  tools 

(11)  Implementation  issues 

—  object  oriented 

(12)  Ada  usage 

—  e.g.,  DMSO  and  government  approval  for  MEL  being  programmed  in  C++ 

( 1 3)  Electronic  exchange  of  metadata 

(14)  Timeframe  schedules  for  current  repository  development 

Priorities  of  Efforts  for  the  Next  Year 

( 1 )  Repository  Subgroup  Charter 

(2)  Interoperability  across  M&S  data  repositories  (i.e.,  CENTCOM,  OASIS,  ARMS,  etc.) 

(3)  Repository  to  support  M&S  data  administration  program  (asap) 

(4)  Electronic  exchange  of  metadata  and  instance  data 

—  standards  to  use  for  exchange  of  metadata  and  data  (new  ones,  existing  ones) 

—  file  formats 

—  exchange  mechanisms 

—  SIG  under  DIS 

(5)  Participate  in  DIS  repository  effort 

Action  Items 

(1)  Insufficiencies  of  DDRS:  brief  DDRS  evaluation  at  meeting  in  mid- August 

—  Due  date:  1  Aug  94 

—  Due  to:  Chien  Huo 

—  Persons  responsible:  Pete  Valentine  and  Mike  Gorman 

(2)  Insufficiencies  of  FIPS  156 

—  Due  date:  TBD 

—  Persons  responsible:  TBD 

(3)  DIS  communication  standards:  exercise  planning,  “lessons  learned” 

—  Due  date:  next  meeting  on  15  August  1994 

—  Person  responsible:  Jim  Watson 
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(4)  Draft  Charter 

—  Due  date:  31  Aug  1994 

—  Persons  responsible:  Jim  Augins,  Linda  Calvert,  Pete  Valentine 

(5)  MITRE  Task  1  Review  (received  today) 

—  Due  data:  22  July  1994 

—  Persons  Responsible:  see  list  of  Subgroup  members 

(6)  MITRE  Task  2  Review 

—  Fax  on  1  August  94 

—  Due  date:  7  August  94 

—  Persons  responsible:  see  list  of  Subgroup  members 

(7)  Status  of  ICASE  and  DoD  Evaluation 

—  Due  date:  15  August  94 

—  Person  responsible:  Linda  Calvert 

6.4  SUBGROUP  ORGANIZATION 

It  was  decided  to  form  another  Subgroup  under  this  TF.  It  will  be  called  the  M&S  Data 
Standardization  Process  Subgroup. 

Co-chairs  are  to  be: 

LtCol  Rick  Barker  (JTAMS) 

Linda  Calvert  (MITRE) 

Other  members: 

Jim  Augins 
Luci  Haddad 
Jim  Hammond 
Mike  Hopkins 
Iris  Kameny 
Eleanor  Schroeder 
Peter  Valentine 
Jim  Watson 

The  goal  of  the  M&S  Data  Standardization  Subgroup  is  to  provide  data  standard  proposal 
packages  to  DoD 

I.  Issues 

1.  Where  to  start? 

a.  C2  Core  Model 

b.  DoD  Data  Model 

c.  M&S  Core  Model  (develop  one) 

d.  Other 

2.  Complex  data  representations  to  include  object-oriented 

3.  External  sources  and  studies 

II.  Requirements 

1 .  Implementing  8320 

2.  Other 
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III.  Pilot  studies  for  information  gathering  and  proof  of  concept 

1 .  Subject  area  information  (S  AI)  model  for  electromagnetics _ 

—  database  and  need  standards  I  proposal  package 

—  connects  to  equipment  bucket  under  Enterprise  model  _ I 

2.  Conventional  Force  Database  IDEF IX  model  address  I  address  how  to  get 

—  in  process  of  reverse  engineering  I  to  proposal  package 

3.  UTSS  reverse  engineering  simulations 
(source  and  requirements  for  target 
simulations)  and  forward  engineering 

—  based  on  airframe  simulators 

—  need  to  integrate  UTSS  model  and  map  to  data  sources 

4.  CCTT  domain  analysis  and  data  model 

5.  SIM  World  DIS  (SPARTA  and  Jim  Watson)  just  beginning  and  may  be  using  IDEF1X 

6.  NWTDB/ARMS: 

—  E/R(?)  models  at  campaign  level 
—  collect  information  on  datn  elements 
—  IDEF  IX  data  modeling 

7.  E2DIS/JTAMS 

IV.  Products 

1.  Policies  and  procedures 

2.  Tools  and  facilities  (i.e.,  repositories) 

3.  Exchange  standards 

6.5  FUTURE  MEETINGS 

The  next  meeting  of  the  Repository  Subgroup  will  be  on  August  15-16,  1994  at  NRaD  in  San 
Diego.  Jim  Augins  will  make  the  meeting  arrangements. 

There  is  no  scheduled  meeting  for  the  Data  Standards  Task  Force  at  this  time  nor  for  the  new 
M&S  Data  Standardization  Process  Subgroup.  The  paper  on  DIS  Needs  for  DoD  Data 
Standards  was  accepted  and  will  be  presented  at  the  DIS  September  meeting  by  Walt  Swindell 
and  Iris  Kameny. 


I  need  to  develop 
1  proposal  package 


I  need  to  develop 
I  proposal  package 
I 
I 


-48- 


7.  SUBGROUP  ON  DATA  VV&C  GUIDELINES  MEETING  NOTES 
7. 1  AGENDA 

THURSDAY  JULY  14,  1994,  1300-  1700 

1300-1330  Review  of  progress:  Bob  Hartling  and  Mark  Ralston 
1330-1400  DIS  Applications  Data  VV&C  Process:  Bob  Hartling 
1400-1445  Non-DIS  Applications  Data  VV&C  Process:  Mark  Ralston 
1445-1500  Break 

1500-1530  Joint  Tactical  Missile  Signatures  Program:  LtCol  Barker 
1530-1545  Summary  of  IEEE  Metadata  Workshop:  Jeff  Rothenberg 

1545-1600  Producer  Data  VV&C  Process  -  What  Needs  to  be  Done?  Bob  Hartling 
1 600- 1 700  General  Discussion  and  Wrap-Up 


DATA  W&C  TASK  FORCE  MEI 

THURSDAY,  JULY  14, 1994 
1300—1700 
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7.3  DATA  VV&C  GUIDELINES  ISSUE  DISCUSSION 

Bob  Hartling  and  Mark  Ralston:  Review  of  Progress  followed  by  DIS  and  Non-DIS  Data 
VV&C  Process 

The  meeting,  co-chaired  by  Mr,  Robert  Hartling,  CNO,  and  Mr.  Mark  Ralston,  AMSAA,  was 
held  on  14  July  1994  at  the  Institute  for  Defense  Analysis,  Alexandria,  Virginia.  The  meeting 
continued  on  the  following  afternoon,  15  July.  The  objectives  of  the  meeting  were  to  review 
progress  to  date  and  determine  future  actions  for  the  subgroup. 

Mr.  Hartling  opened  the  meeting  with  a  review  of  progress  to  date.  The  definitions  of  VV&C, 
developed  by  the  group,  were  reviewed.  These  definitions,  as  shown  below,  have  been  submitted 
to  the  DMSO I/DBTWG  for  acceptance  as  standard  descriptions  of  the  Data  VV&C  process. 

Mr.  Hartling  then  briefed  the  work  that  has  been  accomplished  to  date  on  the  DIS  VV&A/VV&C 
process  flow  diagram  and  “quick  planner”.  After  much  discussion  it  was  agreed  that  the  DIS 
process  flow  diagram  woulr’  be  modified  prior  to  acceptance  by  the  subgroup.  The  DIS 
VV&A/VV&C  process  flow  diagram  was  discussed  again  on  the  following  afternoon.  Mr. 
Ralston  followed  with  a  presentation  of  a  draft  Data  VV&C  process  flow  diagram  for  Non-DIS 
applications.  After  discussion  it  was  concluded  that  this  process  flow  diagram  did  not  adequately 
represent  generic  Data  User  VV&C  processes  and  would  have  to  be  reworked.  Mr.  Ralston  was 
followed  by  LtCol  Barker,  who  presented  a  briefing  on  the  Joint  Tactical  Missile  Signatures 
Program  and  the  data  VV&C  processes  followed  by  this  program.  Finally,  Mr.  Jeff  Rothenberg 
presented  a  report  on  the  first  IEEE  Metadata  Workshop. 

After  the  planned  presentations,  the  group  discussed  approaches  to  future  efforts.  As  a  result  of 
these  discussions  the  group  concurred  with  a  list  of  products  that  would  result  from  future  efforts 
(see  “Products”  below).  In  order  to  produce  these  products,  the  group  decided  to  concentrate 
their  next  efforts  on  the  development  of  a  process  flowchart  and  guidelines  for  User  Data  VV&C 
for  Non-DIS  applications.  In  order  to  accomplish  this  a  plan  to  develop  a  User  Data  VV&C 
process  flowchart  was  developed  and  discussed  (see  “Proposal”  below).  The  group  concurred 
with  this  approach  and  the  meeting  was  adjourned. 

Products 

•  Definitions  —  completed  at  Data  VV&C  TF  meeting  on  April  19,  1994: 

Producer  Data: 

Producer  Data  Verification:  The  use  of  techniques  and  procedures  to  ensure  that  data  meets 
constraints  defined  by  data  standards  and  business  rules  derived  from  process  and  data 
modeling. 

Producer  Data  Validation:  The  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  within  stated  criteria  and  assumptions. 

Producer  Data  Certification:  Determination  by  the  data  producer  that  data  have  been  verified 
and  validated. 

User  Data: 

User  Data  Verification:  The  use  of  techniques  and  procedures  to  ensure  that  data  meets  user 
specified  constraints  defined  by  data  standards  and  business  rules  derived  from  process  and 
data  modeling,  and  that  data  are  transformed  and  formatted  properly. 


-51- 


User  Data  Validation:  The  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  as  appropriate  for  use  in  an  intended  M&S. 

User  Data  Certification:  Determination  by  the  application  sponsor  or  designated  agent  that 
data  have  been  verified  and  validated  as  appropriate  for  the  specific  M&S  usage. 

•  User  Data  VV&C 

—  DIS  applications 

Incorporation  into  process  flowchart 
Incorporation  of  steps  into  quick  planner 

—  Non-DIS  applications 

Process  flowchart 
Guidelines 

•  Producer  Data  VV&C 

—  Process  flowchart 

—  Guidelines 

—  Tools  and  techniques 

—  Quality  prof  ile 

Proposal  to  Develop  User  VV&C  Process  Flowchart  for  Non-DIS  Applications 

I.  Decide  what  the  Non-DIS  data  VV&C  process  model  should  represent 

A.  What  encompasses  Non-DIS?  Answer:  don’t  exclude  initially,  get  examples  from 
Services,  and  decide  if  they  are  appropriate. 

B.  What  perspective  do  we  want  to  represent?  Answer:  model  from  a  data  user  perspective. 

C.  What  functions  (if  any)  do  we  want  to  exclude?  Answer:  model  the  VV&A  process  as 
well  as  the  VV&C  process. 

II.  Task  the  subgroup  Service  representatives  to  prepare  IDEFO  process  models  of 
representative  non-DIS  user  data  VV&C  processes. 


A. 

Army 

—  Mark  Ralston 

B. 

Navy 

—  Bob  Hartlitig 

C. 

OSD 

—  LtCol  Barker 

D. 

JCS 

—  ? 

E. 

AF 

LtCol  Balombini 

Dl.  Meet  to  compare  IDEFO  process  models,  see  if  they  can  be  integrated  into  a  general  process 
model. 

LtCol  Rick  Barker:  Joint  Tactical  Missile  Signatures  (JTAMS)  Program 
The  purpose  of  this  briefing  was  to  gain  insight  for  a  potential  pilot  VV&C  implementation.  The 
background  of  JTAMS  is  that  detailed  signature  data  is  required  for  new  warning  systems.  The 
AF  Electronic  Warfare  Center  maintains  extensive  signature  data  and  conducted  a  1988  survey 
of  missile  warning  system  developers’  needs,  The  result  of  the  survey  showed  that  the 
community  efforts  are  uncoordinated,  standards  are  lacking  and  there  is  a  51%  duplication  of 
tests.  Data  is  not  transferable,  difficult  to  access  and  poorly  documented.  The  JTAMS  solution 
is  to  create  standards  and  a  joint  data  library  in  which  the  data  can  be  verified  with  testing.  The 
JTAMS  program  objectives  are  to  (1 )  develop  joint  standards  for  describing  Tactical  Missile 
Signatures  (TMS);  (2>  develop  joint  standard  procedures  for  acquiring  TMS;  (3)  develop  a  Joint 
Interactive  Data  Library  to  promote  continued  acquisition  and  dissemination  of  TMS;  and  (4) 
develop  a  plan  to  provide  for  formalized  DoD  implementation  of  TMS  standards  and  to  ensure 
future  utilization  of  the  Data  Library  by  all  the  services.  JTAMS  benefits  are  for  measurement 
community  improvements,  high  quality  data  for  the  acquisition  process,  improvements  in  data 
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archiving,  and  model  improvements,  Library  will  be  Oracle  Based  (Joint  Electromagnetic 
Signatures  and  Effects  Database  Library  (JESEBEL))  and  will  do  complete  signature  data 
management  using  modular  design  methodology.  The  data  quality  implications  are:  there  is  no 
such  thing  as  “best”  data  since  correctness  and  completeness  are  always  relative  to  a  particular 
purpose,  and  cost-benefit  analysis  is  necessary  to  decide  how  much  data  V&V  is  worth  doing 
and  how  much  model  V&V  is  worth  doing.  JESEBEL  is  a  representation  of  abstract  TMS  and 
there  could  be  many  different  representations.  JTAMS  promotes  TMS  quality  by  recording 
sufficient  metadata  about  TMS;  by  controlling  and  improving  the  processes  that  affect  TMS;  by 
performing  producer  and  consumer  VV&C  in  terms  of  its  metadata  which  specifies  its  intended 
users  and  purpose;  and  by  recording  the  TMS  evaluation  results  in  additional  metadata. 

Mr.  Jeff  Rothenberg:  Summary  of  IEEE  Metadata  Workshop  held  at  National  Archives 
May  16-18, 1994 

The  workshop  focused  on  scientific  data  and  metadata  needs:  need  to  share  data  across 
disciplines  and  contexts  where  data  may  appear  different  in  different  disciplines  or  contexts; 
scientific  data  may  have  special  characteristics  (effect  of  instrumentation,  huge  datasets);  and 
they  need  to  produce  a  reference  model  or  framework  for  scientific  metadata.  The  metadata 
needs  to  be  machine  interpretable,  should  be  useful  in  allowing  systems  to  help  users,  should  be 
human-readable,  and  a  standard  representation  for  metadata  is  highly  desirable.  Many  aspects  of 
data  must  be  represented:  non-scientific  domains,  non-scientific  attributes  of  data  such  as 
ownership,  privacy,  cost,  quality,  usage,  etc.;  and  the  metadata  has  to  serve  many  different  uses. 
They  are  beginning  by  trying  to  find  a  suitable  modeling  formalism  and  then  applying  that 
formalism  to  a  number  of  contexts  to  be  able  to  derive  a  general-purpose  metadata  model. 

The  DoD  8320.1-M.3  Draft  document:  DISA/JIEO/CIM  furnished  a  draft  “Data  Quality 
Assurance  Procedures”,  February  1994,  is  included  in  Appendix. 

7.4  SUBGROUP  ORGANIZATION 

There  is  no  change  in  the  Subgroup  organization. 

7.5  FUTURE  MEETINGS 

The  next  meeting  of  the  Data  VV&C  Guidelines  Subgroup  was  tentatively  planned  for  18 
October  1994. 
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8.  DATA  VV&C  TASK  FORCE  MEETING  NOTES 
8.1  AGENDA 

FRIDAY  JULY  15,  1994,  0800-  1700 

0830-0845  Review  of  goals  and  progress:  Iris  Kameny 

AUTHORITATIVE  DATA  SOURCES  SUBGROUP 


0845-0915 
0915  -0945 

0945-1000 

1000-1030 

1030-1100 

1100-1130 

1130-1200 

1200-1300 


Summary  of  prior  meeting  and  comments  on  M&S  data  categories:  Bill  Dunn 

Data  aggregation,  security  issues,  release  authority,  and  recommendations: 

Allen  Hess 

Break 

Populating  the  data  s<  urces  listings  format,  who  does  what  and  when:  Bill  Dunn 

Authoritative  data  sources,  who  are  they,  data  contents,  data  provider,  users 

Data  centers  -  who  are  they,  what  authority  do  they  have,  data  contents,  providers 
and  users  responsibilities:  George  Flax  (but  he  didn’t  present) 

Requirements  for  the  next  meeting:  Mike  Hopkins 

Lunch 

DATA  VV&C  GUIDELINES  SUBGROUP 


1300-1600 

1600-1700 


VV&C  guidelines  and  quality  profile  working  session:  led  by  Bob  Hartling  and 
Mark  Ralston 

Wrapup:  Iris  Kameny 


8.1  ATTENDEE  LIST 
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8.3  DATA  VV&C  ISSUE  DISCUSSION 
Iris  Kameny:  Review  of  Goals  and  Process 

Iris  briefly  presented  the  long  term  goals  and  goals  of  the  subgroups  to  bring  everyone  up-to- 
date.  This  was  brief  since  most  had  attended  the  Data  VV&C  Guidelines  Subgroup  meeting  the 
previous  day.  The  briefing  is  in  the  Appendix. 

She  stressed  Chien  Huo’s  request  for  a  mission  statement  for  the  task  force  and  each  of  the 
subgroups. 

Mr.  Bill  Dunn:  Summary  of  Prior  Meetings  and  Comments  on  M&S  Data  Categories 
There  is  a  confusion  about  data  centers  that  may  not  be  authoritative  data  sources  (e.g.,  TRAC 
TADS)  and  about  authoritative  data  sources  (ADSs)  that  may  not  have/manage  any  instance  data. 
What  is  an  authoritative  data  source?  It  is  a  source  that  is  internal  to  a  Service  and  it  is  selected 
or  pointed  to  by  the  Service  as  an  authority  for  a  particular  category  of  data.  Centers  may 
validate  data  01  just  verify  data  they  have  accumulated  and  modified.  The  data  taxonomy  shows 
leaf  nodes,  each  representing  a  data  category  for  which  there  could  be  an  authoritative  data 
source. 

Allen  Hess:  Data  aggregation,  security  issues,  release  authority,  and  recommendations 
Release  authority  issues:  (1)  the  definition  of  release  authority;  (2)  data  centers  must  have 
reasonable  authority  over  data  they  manage;  (3)  originating  authority  for  release  of  specific  data 
may  not  wish  to  release  the  control  over  the  data;  and  (4)  a  user  who  has  need  to  know  and  meets 
all  other  requirements  for  examining  the  data  must  be  able  to  gain  access  to  the  data.  The 
Appendix  contains  the  briefing  which  goes  into  detail  on  the  definition  of  release  authority  in 
DoDD  5230.24/.25.  Hess  notes  that  security  guidance  deals  well  with  classification  of  data  but 
not  with  distribution.  It  is  the  distribution  statement  that  is  missing  for  datasets  that  makes 
release  so  difficult  to  accomplish.  He  suggests  that  the  ADS  guidelines  should  include  a 
statement  to  the  effect  that  the  review  process  at  the  data  center  must  ensure  there  is  a 
distribution  statement  agreed  to  by  the  original  authority.  Currently  at  BMDO,  data  can  be 
transferred  to  a  data  center  manager  without  transferring  release  authority  which  essentially 
prevents  data  center  personnel  and  users  from  accessing  the  data.  When  attempting  to  obtain 
release  authority,  the  originating  authority  (study  director)  may  no  longer  be  available  and/or  it 
may  involve  a  lengthy  process.  Note  that  when  data  is  released  to  a  data  center,  it  is  really 
released  to  the  authority/responsibility  of  the  data  center  director  but  not  to  others. 

How  do  classified  data  centers  exchange  data?  Again  this  requires  a  distribution  policy  and 
procedures.  Lessons  learned:  (1)  Directive  No.  3240  should  require  that  release  authority 
accompany  archive  deliverables;  (2)  each  program  should  compile  a  list  of  users  with  need  to 
know;  (3)  program  classification  guidance  must  be  on  file  at  the  data  center  and  must  contain 
distribution  statement  information;  (4)  use  a  test  case  to  establish  release  authority  standards;  (5) 
develop  standard  forms  for  user  data  requests  and  release  authority;  (6)  it  is  easier  to  deal  with 
release  authority  if  data  is  organized  by  programs;  and  (7)  data  centers,  study  directors, 
information  systems,  security,  etc.  must  all  participate  in  release  authority  solutions. 

Also,  release  authority  for  experiment/test  data  must  address  catalog  information,  metadata  and 
data  since  the  release  authority  may  be  different  for  these  different  types  of  data. 

With  respect  to  data  aggregation:  Hess  suggests  that  a  methodology  should  be  designed  to 
determine  the  probability  that  one  will  need  to  do  a  detailed  examination  of  data  aggregation 
issues.  There  could  be  an  algorithm  to  determine  the  need  for  re-examining  the  classification 
level  of  a  data  aggregation  or  dis-aggregation.  Putting  two  data  elements  together  is  probably  not 
a  problem.  But  if  you  put  50  additional  pieces  of  data  into  a  threat  environment  where  there 
already  is  classified  data,  then  that  may  be  a  different  story. 
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The  Navy  takes  data  from  a  highly  classified  source,  aggregates  the  data  and  releases  it  at  the 
unclassified  level  removing  any  identification  of  the  originating  source(s). 

John  Coale  told  of  a  court  case  where  DIA  said  they  couldn’t  release  aggregated  data  at  the 
unclassified  level  even  though  the  individual  data  was  unclassified.  However,  the  court  told 
them  to  release  it  as  unclassified  a  single  page  at  a  time. 

In  summary:  we  need  to  build  standards  for  release  distribution  into  the  process  in  order  to  make 
data  releasable  and  shareable. 

Mark  Ralston:  Authoritative  data  sources,  who  are  they,  data  contents,  data  provider, 
users  responsibilities,  and  “data  on  demand”. 

An  authoritative  data  source  (ADS)is  an  organization  designated  and  recognized  as  the  producer 
of  best-estimate  values  for  one  or  more  categories  of  data  or  an  authoritative  data  source  is  an 
organization  designated  to  conduct  producer  VV&C  activities  for  one  or  more  categories  of  data. 

Responsibilities  of  an  ADS  include:  (1)  maintaining  subject  matter  expertise  in  those  technical 
areas  that  relate  to  the  data  categories  that  the  organization  is  responsible  for;  (2)  data  centers 
should  be  producing  data  to  meet  the  needs  of  data  users  either  by  filling  the  repository  in 
anticipation  of  needs  or  producing  data  “on  demand”;  (3)  ADSs  conduct  producer  V&V;  (4) 
ADSs  prepare  data  producer  certification;  and  (5)  ADSs  develop  and  maintain  appropriate 
database  technology  to  meet  the  needs  of  data  centers  and  data  users  (i.e.,  data  standards  and 
timoliness). 

AMS  AA  has  been  delegated  to  be  an  ADS  for  equipment  performance  and  characteristic  data  by 
AMC.  Its  mission  is  to  serve  as  an  independent  technical  evaluator  for  major  systems;  perform 
materiel  system  analyses;  perform  logistics  analyses;  serve  as  a  VV&C  authority  for  equipment 
performance  data;  and  for  model  development^ V& A. 

TRAC  serves  as  a  data  distribution  center  (repository)  for  data  users  within  TRADOC  with 
AMSAA  producing  new  data  only  when  necessary.  Much  of  the  data  AMSAA  produces  is 
model  output  using  input  variables  that  are  often  unique  for  a  specific  user’s  need.  This  requires 
the  ability  to  generate  data  of  a  recognized  “standard"  type  with  different  input  variables  “on 
demand”  to  support  a  single  data  request. 

AMSAA  has  ~  100  people  involved  in  data  VV&C,  while  TADS  has  only  10-15  people. 

Mike  Hopkins:  Requirements  for  next  meeting 

DMSO  IssuesAThings  to  Do  for  ADS  and  ADS  Process: 

(1)  IDEFO  models  for  Services  VV&A  “as-is”  process  are  needed.  There  may  be  many  of 
these  per  Service  (depending  on  how  many  different  ways  the  VV&A  process  is  carried 
out).  Especially  need  support  on  integrating  IDEFO  models  (may  need  AI  support  for 
this). 

(2)  Need  a  data  model  for  an  Authoritative  Data  Source  Directory 

(3)  Need  to  address  security  issues,  in  particular,  data  aggregation  and  release  authority. 

(4)  Need  to  complete  the  ADS  guidelines  document. 

Things  to  Do  for  the  Next  ADS  Meeting: 

(1)  Nail  down  the  data  category  matrix:  everyone  send  comments  to  Walt  Swindell  and 
Howard  Haeker  and  let’s  go  another  round  with  it  and  get  a  quick  response. 
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(2)  Need  subset  of  data  center  people  to  look  at  that:  CENTCOM,  TADS,  OASIS,  ARMS, 
AF  needs  to  nominate  a  data  center,  and  NWTDB. 

(3)  Answers  to  who  are  the  ADS,  where  are  they,  what  are  their  responsibilities  and  how  do 
they  connect  to  producers  and  users:  get  DISA  or  OSD  survey  of  data  centers. 

Bub  Hartling  and  Mark  Ralston:  VV&C  Guidelines  Meeting  (continued  from  previous 
day) 

The  study/exercise  directors  make  decisions  as  to  M&S  VV&A.  DIS  VV&A  working  group  is 
coming  up  with  recommended  studies  for  VV&A.  The  Components  are  defining  their  VV&A 
processes.  AR  25-9  discusses  VV&A  for  the  Army. 

Issues  in  Response  to  Chien’s  Request: 

(1)  Pilot  studies  for  VV&C  Guidelines:  should  be  5-6,  need  to  pick  the  right  diversity  of 
projects 

(2)  AI  tool  support  for  VV&C  using  quick  planner  for  non-DIS 

(3)  DIS  exercise  director’s  tools  to  coordinate  data  for  all  M&S  used  in  an  exercise  (think 
that  ARPA  is  doing  this  for  DIS) 

(4)  For  producer  data  VV&C  process  models,  suggest  using  DMA  and  Navy  Oceanographic 
databases  as  examples 

Simone  Youngblood:  DISW&A 

DIS  exercise  takes  place  at  the  entity  level  and  provides  everyone  with  ground  truth  of  the  virtual 
battlefield.  Every  M&S  has  its  own  data  representations.  They  will  have  a  common  database  for 
an  exercise  (e.g.,  for  terrain  data).  Each  M&S  must  be  VV&Aed  and  each  contains  data  that  is 
not  passed  to  it  by  PDUs. 

DIS  repository:  There  is  a  basic  need  for  a  DIS  repository  to  be  used  when  building  an  exercise. 
It  is  where  the  exercise  director  goes  to  find  pointers  to  M&S  and  M&S  data.  DIS  needs  catalog 
type  of  information.  DIS  needs  a  distributed  directory  and  needs  to  keep  past  VV&A 
information  and  pointers  to  Components  repositories.  The  DIS  exercise  directory  needs  to 
include  references  to:  lay  down,  threat,  V&V,  Component  pointers  to  M&S  and  to  standard 
databases;  and  fidelity  information  (there  is  a  whole  fidelity  taxonomy  that  would  include  the 
level  of  resolution).  The  Components  would  input  data  into  the  repository  when  they  go  through 
compliance  testing  of  M&S.  ( Note,  compliance  testing  only  assures  that  they  are  producing 
correct  PDUs,  it  says  nothing  about  validity.)  Past  exercise  reports  wouid  also  go  into  the  DIS 
repository. 

8.4  TASK  FORCE  ORGANIZATION 


No  change  in  the  Data  VV&C  Task  Force  organization. 

8.5  FUTURE  MEETINGS 


Next  meeting  of  the  Authoritative  Data  Sources  Subgroup  will  be  held  on  September  9,  1994 
at  IDA. 

Next  meeting  of  the  Data  VV&C  Task  Force  and  the  VV&C  Guidelines  Subgroup  will  be  held 
on  October  18-19,  1994  at  IDA. 
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LTC(P)  Jerry  Wiedewitsch,  USA 
Deputy  Director,  DMSO 


platforms 


Major  Forces 
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Exercise  in  Confederated  Simulation 


Synthetic  Theater  of  W ar  [  PMSO 
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[  DMSO  1 


presented  at  the 


Eighth  Information/Data  Base 
Technology  Working  Group  (IIDBTWG) 
Conference 


i 

Data  Administration  I 


11*12  July  1994 


DMSO 

1901  N.  Beauregard  Stmt.  Suit*  504 
Alexandria,  Virginia  22311 
(703)  99S-06S0 
(703)  99M«7  fax 


Dr.  Chien  Huo 
DMSO  &  JIEO/CFS 

Ms.  Iris  Kamtny 
Rand  Corporation 


DMSO  Objectives  [  DMSO  ) 


1.  Provide  Technical  Framework  for  Modeling  &  Simulation 

2.  Develop  Authoritative  Representations 

3.  Integrate  Live,  Virtual,  and  Constructive  Simulations 

4.  Broaden  Modeling  &  Simulation  Applications 
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Over  Arching  Purpose  of  the 
DMSO  Data  Related  Activities 


[  DMSO  1 


To  promote  interoperability,  sharing  and  reuse  of  data  and  models 

•  In  coordination  and/or  compliance  with  DISA/JIEO/CIM 

•  Through  data  standardization  and  data  related  efforts  not  being 
addressed  by  the  current  Corporate  Information  Management 
(CIM)  initiative,  or  the  commercial  world 

•  With  participation  and  concurrence  from  the  M&S  community 
through  M&S  projects  and  components’  M&S  offices 


M&S  Functional  Data  Administrator  (FDAd) 

Responsibilities 


[  DMSO  ] 


•  Implement  a  M&S  Data  Administration  (DA)  infrastructure 
and  to  establish  community  consensus  on  policies,  procedures 
and  standards 

•  Address  complex  data  standardization,  data  verification, 
validation  and  certification  (W&C) 

•  Establish  a  M&S  repository  and  the  development  of 
taxonomy,  Databases  Directory  and  M&S  Directory 

•  Identify  and  promulgate  DA  methodology  and  tools 

•  Facilitate  interchange  of  information  and  lessons  learned 
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Near  Term  Actions  [  DMSO  ) 


•  Develop  Guidelines  for  data  standardization  policies  and  procedures 
including  complex  data  and  VV&C  for  M&S  community 

•  Oversee  task  groups  addressing  complex  data,  M&S  data  staudards,  data 
VV&C,  and  DMSO  funded  database  projects 

•  Develop  a  prototype  M&S  repository  built  upon  the  Defense  Information 
Repository  System  (DIRS)  &  Industry  Standards  to  support  DA 
standardization 

•  Develop  M&S  Directory  (catalog)  and  Database  Directory  (catalog)  and 
Authoritative  Data  Sources  Directory 

•  Support  information  sharing,  issue,  definition  and  pursuit: 

-  I/DBTWG 

-  Task  Forces 

•  M&S  Information  System  (M&SIS) 


I/DB  TWG  Structure, 

Task  Forces,  and  Subgroups  (  DMSO  ) 


I/DB  Technology  Working  Group 


Complex  DtU 

iMfcifmt 

Catetortzatlon 
—  and  guidelines 
(JDBE,  MITRE) 

_ Taxonomy 

mmj-4) 


I  Complex  data  pilot  itudlet 


Data  W&C 

iMKEsra 


—  Quality  profile 
and  (uidtilna 
(NtllD.AMSAA) 

—  Authoritative  data 
source* 

(MISMA,  CENTCOM) 

—  WJtC  pilot  studies 


Data  Standards 
FmK  Fartt 

_ Repositories 

( NRaD ) 

DISdaU 
—  standards 

(TRAC) 


«WW0“*** 
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M&S  Data  Administration 
Policy,  Procedures  and  Standards 


[  DMSO  ) 


«  Data  Standardization 

-  Developing  a  Data  Administration  Strategic  Flan  (DASP) 

-  Rapid  Data  Standardization  Guidance 

*  Major  areas  addressed 

-  Policy,  procedures,  tools  fur  modeling  and  standardizing 
complex  data 

-  Plans  for  developing  M&S  repository  to  contain 

.  Complex  data  standards 

.  Domain  nomenclature  and  symbology  standards 

.  Tools  to  manipulate  and  interoperate  between  repository  objects 

-  Guidelines  and  responsibilities  for  Authoritative  Data  Sources 

-  Guidelines  and  tools  for  carrying  out  Data  W&C  including  a 
database  quality  profile 


jfiSk  M&S  Directory  (Catalog)  and 

Data  Base  Directory  (Catalog)  [  DMSO  ) 


*  Data  Models  for  both  directories  will  be  available 
for  reuse  throughout  M&S  community 

•  Data  models  include  taxonomies  to  aid  in  browsing 
and  access 

•  Prototype  impSem&itatton  to  be  completed  by  early 
calendar  year  1995  ( with  some  population  of  directories) 

*  World  Wide  Web’s  MOSAIC  as  a  browsing  and  query  tool 
starting  March  1995 
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I/DB  Portion  of 
M&S  Information  System 


[  PMSO  ) 


*  Calendar  and  agendas  for  future  I/DB  and  related  Task 
Force  meetings 

*  Notes  from  previous  meetings 

*  Special  sections  for  Task  Force  (Complex Data,  Data  Standards, 
Data  W&C,  Repository)  information 

*  Membership  list 

*  Information  technology  relevant  definitions,  acronyms, 
and  references 


Data  VV &C  Task  Force 
Long  Range  Objectives  [  PMSO  ] 


•  Develop  guidelines  for  performing  Data  W&C  coordinated 
with  W&A 

-  Definitions  and  process 

-  Cost  models  and  cost  information 

-  Quality  profile  and  metadata  definitions 

•  Address  authoritative  data  sources  and  their  responsibilities 

•  Address  role  of  M&S  data  centers  between  data  sources 
and  simulation  centers 
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Authoritative  Data  Sources 
Draft  Paper 


DMSO 


Draft  paper  “Authoritative  Data  Sources  for  DoD  Modeling 
and  Simulation  Applications”  (April  19, 1994) 

Addresses 

-  Service  agency  names  and  authoritative  sources 
according  to  mission  functionality 

•  Data  Centers  with  customers  and  functionalities 
they  serve 

•  Potential  opportunities  for  sharing  and  reuse 

-  Responsibilities  of  data  centers  and  data  customers 


Complex  Data  Challenges  [  DMSO  ] 


*  Definition  (Preliminary,  from  the  April  6-7, 1994  meeting  of  the 
Complex  Data  Categorization  Subgroup) 

-  Complex  data  is  that  data  which  is  difficult  or  awkward 

to  model  using  commonly  used  techniques  (Le.,  IDEF1X  and 
other  kinds  of  Entity-Relationship  modeling) 

•  Dependencies 

-  Active:  Age 

-  Genealogy  audit  trail:  Radar  Pulse- Repetition  Interval  (PRI) 

•  Mapping  (umaps  tolfrom ”) 

-  Inter-model:  Radar  PRI 

-  Intra-model:  Age 

-  Derivation:  Radar  Pulse-Repetion-Frequency 

*  Artifacts  of  legacy  systems  and  cost  constraints 

-  Multipurpose:  e.g.,  Vehicle-Capacity 

-  Coupled:  e^  Sex-Marital-Code 
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Summary 


•  M&S  Data  Administration/Standardization  Program  is  a 
developing  area. 

•  Need  your  assistance  to  facilitate  M&S  community  in 
data  standardization 


Points  of  Contact 


DMSO 


•DMSO  PM  for  Data  RaiMad  ActlrlUaa:  CDR  Garr  MUeh 

Phanai  (713)  m-ttM  Eio  nil; 

•MAS  Dtu  AdudDiitralioa:  Dr.  CnUa  Hu 

PtMMU  (7*3)I»M4*I  Email: 

•MAS  Diractorr  and  D*  Directory  Data  Mudala;  Mr.  Stara  Mauuur* 

Phcrnn  (4«)i3M4*7  EmalL 

•MAS  Director?  a»4  DR  Director,;  DMukmtnlallon:  Or.  Mika  Fru» 

Phoaas  (7*1)  MS-Ml  Email: 

•Accaaa  la  MAS  Iofareuittoa  Syattm-  Mr.  Kan  Whnmar 

rtena:  (7«)27M77*  Email: 

-Camala  Data,  Data  Standard*  aad  Dau  VVAC  Talk  Parma:  Ml.  Lru  Kai 

Pboaai  Oil)  3*3-141 1  EaaUt 
*7174 

Camalaa  Data  CalaaarinUad  Subgroup:  Dr.  L»  Sdlfman 

Phaa*  (7«3)07-»75  Email: 

•Taxoaomjr  Sahraap:  Ma.  Irta  Kamaa;  laud  UCei  Did  »!<**> 

Pkoaa:  (311)  Ill-Mil  Email: 

*7174 

•WAC  GaldaUaaa:  Mr.  Dab  Hartllat  add  Mr.  Mt-k  Ualtlaa 

KarfUdf  Pfcoaa:  (713)  MM7I1  datum  P 

•AultwrMatlya  Data  Saarcau  Mr.  till  Dunn  and  M.  Mark  Hopklai 

Duaa  Phaaa:  (7*3)  4I7-S3EC  Hapklai  I 

•Rapaaltartaai  Mr. Jtaa  Auplna  PW  <M«)  SS3**M9’£aaU: 


Email;  |tal*r*j#daaaadUc^lajnll 


k  mc#dmio.dllc.illr mil 

amiMuw4hmcliuca-amlil7.arinjr.aUI 

frMM#ldA«rf 

kwimiar#dnu«  dllr.dla.mil 

kamaajr#raa4ar| 


EauU:  taUgaMnCmlira^rt 
Email:  bamjenadari 


>h*aai  (411)  371- 3344 

Phoud  (113)  IV-4431,  ill* 
amliu4aaac.mil 
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(  DMSO  ) 


1 


Backups 


Draft  Taxonomy  for 
M&S  Directory 


[  DMSO  1 
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Definitions  of  Data  W &C  f  DMSO 


W&C  Definitions  (April  19, 1994)  from  Data  W&C  Task 
Force  Meeting  at  IDA 

Producer  Data 

-  Producer  Data  Verification 

the  use  of  techniques  and  procedures  to  ensure  that  data  meets 
constraints  defined  by  data  standards  and  business  rules  derived 
from  process  and  data  modeling 

•  Pcodut£LD.aia.Y.alldaliQQ 

the  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  within  stated  criteria 
and  assumption 


determination  by  the  data  producer  that  data  has  been  verified 
and  validated 


Definitions  of  Data  W &C  f  DMSO 


continued 

•  W&C  Definitions  (April  19, 1994)  from  Data  W&C  Task 
Force  Meeting  at  IDA 

•  User  Data 

•  .User  .Data  Vtrificutifln 

the  use  of  techniques  and  procedures  to  ensure  that  data  meets  user 
specified  constraints  defined  by  data  standards  and  business  rules 
derived  from  process  and  data  modeling,  and  that  data  are  transformed 
and  lormatted  properly 

•  UscrJPntaYiiliiMnn 

the  documented  assessment  of  data  by  subject  area  experts  and  its 
comparison  to  known  or  best-estimate  values  as  appropriate  for  use  in 
an  intended  M&S 


determination  by  the  application  sponsor  or  designated  agent  that  data 
have  been  verified  and  validated  as  appropriate  for  the  specific  M&S 
usage 

flays™1-*4* 
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Authoritative  Data  Sources 
Problem 


[  DMSO  ) 


•  No  Service  organization  is  in  place  to  serve  as  comprehensive 
authoritative  data  source  for  that  Service 

-  Authorized  sources 

result  of  instructions  or  directives  that  authorize  funding  for 
organizations  and  describe  in  detail  what  the  organization  is  to  do, 
what  areas  it  has  cognizance  over,  and  what  authority,  it  has 
DMA,  DIA) 

-  Pe  facto  sources 

become  authorities  because  of  the  information  they  possess 
(e.g.,  Air  Foret  Logistics  Command  at  Wright  Patterson,  Ships  Parts 
Control  Confer  at  Mechanics  burg  PA,  Army  TR.KDOC  Analysis  Center 
(TRAC)  at  Ft  Leavenworth ) 

•  Factors  contributing  to  multiple  authorities: 
resolution  and  aggregation,  classification 

ruw™"-* *•  19 


Authoritative  Data  Sources 
Responsibilities 


•  Use  of  data  modeling  and  data  standards 

•  Carrying  out  of  data  YV&C 

«  Configuration  management  of  data  and  processes 
producing  data 

•  Help  to  M&S  users  of  data 

•  Handling  of  data  security  issues  such  as  data  aggregation 

•  Participation  in  M&S  VY& A 


[  DMSO 
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Complex  Data  Categorization  [  DMSO  ) 


Definition  of  complex  data  (Preliminary) 

Complex  data  is  that  data  which  is  difficult  or  awkward 

to  model  using  commonly  used  techniques 

(i.e.,  IDEF1X  and  other  kinds  of  Entity-Relationship  modeling) 

from  the  April  6—7, 1994  meeting  of  the  Complex  Data  Categorization  Subgroup 


21 


Complex  Data  Challenges  [  DMSO  ) 


•  Dependencies 

•  Active:  e^.,  update  Age  each  time  Current-Date  changes 

•  Genealogy  audit  trail:  Radar-Pulse-Repetition-Interval  conies  from 
EWIR  version  1.5  last  changes  on  01  Jan  94 

•  Mapping  ("maps  tolfrom”) 

•  Inter-model:  Radar-PRI  in  Model  A  is  the  same  as 
Radar-Pulse-Interval  in  Model  B 

-  Intra-model:  Age  depends  on  Current-Date  and  Birth-Date 

•  Derivation;  Pulse-Repetition-Imerval  in  Model  A  is  the  reciprocal 
of  Pulse-Repetition-Frequency  in  Model  B 

•  Artifacts  of  legacy  systems  and  cost  constraints 

-  Multipurpose:  e.g.,  Vehicle-Capacity  may  contain  either  number  of 
passenger  or  weight  carrying  capacity  depending  on  vehicle  type 

•  Coupled:  e^.,  Sex-Marital-Code  bundles  information  on  a  person’s 
Sex  and  Marital-Status  in  a  single  data  element 


m i™*** 
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Categories  of  Data 
meeting  the  definition  of 
Being  “Complex”  (N on-Exhaustive) 


fDMSO") 


•  Inheritance  (“is-a”) 

a  relationship  in  which  one  or  more  subclasses  “inherit”  ail 
the  attributes  and  relationships  of  their  super-classes 

•  Composition  (“has  a”,  “is  a  part  of*) 

data  entities  comprising  instances  of  data  which  relate 
to  other  instances  of  data  within  the  same  entity 

«  Derivations  (“comes  from”) 

data  that  is  computed  from  other  data  that  is  stored  in 
the  same  or  in  other  databases 


Raff™0*** 


Inheritance  (“Is-A”)  (  dmso) 


Vehicle 

Is-a'  'le-A 

Wheel  Vehicle  Track  Vehicle 

•  IDEF1X  supports  inheritance  using  the  concept  of  category 

•  Complex  data  need  more  powerful  notions  of  inheritance 

•  Multiple  inheritance 

given  class  has  multiple  super  classes:  e,g.,  grad- student-assistant  is 
subclass  of  student  and  employee  both  of  which  are  subclasses  of  people 
-  Multiple  Is-A  hierarchies 

hierarchies  have  no  common  root  (tank  is  a  vehicle  and  tank  is  a  weapon) 

•  Polymorphic  attributes 

an  attribute  has  different  interpretations  within  Subclasses  of  a  common 
Is-A  hierarchy  (VehiclfS peed  expressed  in  different  units  of  measure  for 
different  kinds  of  vehicles) 

Raft”'0-"* 
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Composition 

(“has  a”,  “is  a  part  of’)  [  DMSO  ] 


Composition  M&S  data  is  awkward  to  represent  using  relational 
or  IDEF1X  concepts 

•  Directed  graphs  such  as  command  hierarchies,  bill  of  parts 

-  Construction/complex  structures 
such  as  road  networks,  compound  documents 

•  extensible  set  of  data  types 

e.g.,  binary  large  objects  (BLOBS) 

•  chains 

e.g.,  address  made  up  of  street,  city,  state,  code 

•  other  types  of  construction 


Raft™.*.** 


Derivations 

(“comes  from”)  e.g.,  age,  Pk  [  DMSO  ] 


Algorithms 

-  Within  instances 

e.g.,  Age  of  a  single  PERSON 

*  Across  instances  (aggregations) 

Average-Age  of  ail  PERSONS 

•  Stated  explicitly 
c.g.,J:^X**2+Z 

-  By  reference 

using  Euler’s  equations 

Le.,  the  internal  workings  of  the  algorithm  are  not  described 


paar10*-* 
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Data  VV&C 

as  part  of  the  DIS  Process  [  DMSO  ) 


Database  Quality  Profile  [  DMSO  ) 


•  Comes  from  VV&C  audit  trail  metadata  from  all  levels 

-  Database  (DB)  level:  certifies  V&  V  applied  to  the  DB 
as  a  whole 

-  Data-element  level:  certifies  V&V  applied  to  data  elements 
and  their  domains 

•  Data  value  level:  certifies  V&V  applied  to  individual 
data  values 

*  V&V  audit  trail  information  is  open-ended 

-  Specifies  who  has  done  what,  when,  to  verify/validate/certify 
the  data 

-  Applies  to  data  values,  metadata  values,  data-element 
definitions,  and  the  DB  as  a  whole 

•  Represents  both  producer  and  user  V&V 
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Reviews  Supported  by  JDBE 


SSvSvflvlvSvtvSSwSR?!????! 


Subject 

Organization 

Action 

Science  dr  Technology  Starter  Set 

USD(A) 

Changes 

Suggested 

Military  Personnel  Conceptual 

USD(P&R) 

Changes 

Data  Model  (5  Submissions) 

Suggested 

C2  Core  Data  Model  (5  Views  and 

DISA 

Changes 

Stakeholder's  Meeting) 

Suggested 

Reviews  Supported  by  JDBE 

(continued) 


Subject  Organization  Action 

Corporate  Logistics  Data  Model  USD(Logistics)  No  Comment 

(Starter  set  and  1  View) 


DoD  Finance  and  Accounting  Comptroller  No  Comment 
Data  Model  (3  Views) 


Geospatial  Data  Model 


DISA  Fending 


Entity  Labeling  and  Definition  DISA 
Guidelines 


Changes 

Suggested 


Page  3 


-  88  - 


BSeE§B 


mprovements  Needed  m 
the  Review  Process 


•  Packages  sent  directly  to  reviewers 
designated  by  the  FDAds  and  CD  Ads 

•  Submissions  sent  via  electronic  media 

•  All  submissions  accompanied  by  a  data 
model 

•  Comments  submitted  via  the  Defense 
Data  Repository  System  (DDRS) 


J 
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DATA  W&C  TASK  FORCI 


-  89  - 
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•  •  • 


ng  Range  Objectives  for  Data  VV&C 

(April  19, 1994)  _ 
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M&S  Data  Categories  (Army/TRAC) 

These  Categories  are  applicable  to  the  US  and  Foreign  Countries 
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Military  Operations  Research  Society  (MORS) 
Simulation  Data  and  Its  Management  (SIMDATAM) 
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Establish  Interface  Issue  Methodology 

Alignment  with  C3I/DISA  Direction 

-  common  tools 

-  methodologies 

-  Dll  shared  Infrastructure 
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C2  FDAd  Mission 


Achieve  a  fully  interoperable  C2  environment 
through  effective  data  standards  coordination 
and  program  development,  including  data 
elements  and  data  models  for  C2  Projects, 
programs,  and  migration  systems. 


07/11/1994 
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07/11/1994 


C2  Functional  Data  Administrator 

(FDAd) 


Ms.  Deborah  Castleman 

Office  of  the  Secretary  of  Defense, 

Deputy  Assistant  Secretary  of  Defense  for 
Command,  Control,  and  Communications  (C3) 


Authority 


■  DoD  Directive  8000.1,  "Information 
Management  Program" 

■  DoD  Directive  8320.1,  "Department  of 
Defense  Data  Administration" 

■  Command  and  Control  Data 
Administration  Strategic  Plan,  FYs 
1994-2001 


Goals 


■  Support  Joint  and  Combined 
Operations 

■  Aggressively  Implement  C2  Data 
Administration 

■  Promote  coordination,  cooperation,  and 
participation  of  Functional  Areas  and 
Components  (C/S/A) 


Functions 


Provide  Standard  Data  Elements 

Maintain  C2  Core  Data  Model 

Integrate  C2  Subarea  Data  Models 

Coordinate  with  Functional  and 
Component  Data  Administrators 
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Support  to  GCCS 


Principal  C2  Migration  System 

Vehicle  for  Accelerating  Data 
Standardization 

Targeted  Focus  Sessions  and 
Collaborative  Modeling 


Page  7 


C2  Data  Administration 
Project  Areas 

■  Data  Modeling 

■  Data  Elements 

■  Data  Administration  Policies  and 
Procedures  (document) 


Data  Modeling 

Process 

-  Subarea  models 

-  Integrate  with  C2  Model 

-  Propose  Changes  to  DoD  Model 
Efforts 

-  C2  Core  Data  Model 

-  Fire  Support  Model 

-Air  Operations  Model  (FPI) 

-  SOCOM  Model  (FPI) 
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07/11/1994 


Data  Elements 

From  the  C2  Core  Data  Model 
C2  Input  to  Starter  Set 
JUDI  Data  Elements 
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Program  Policies  and  Procedures 

(work  in  progress) 

■  Implement  DoD  Data  Administration 
policy  and  direction 

■  Establish  Procedures  for  C2  Data 
Administration 

■  Describe  C2  Configuration 
Management  (CM)  Body 

■  Authorize  publication  of  Configuration 
Management  Procedures 
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Summary 

■  Mission 

■  Goals 

■  Functions 

■  Projects 
-Data  Modeling 

-  Data  Elements 

-  Policy  and  Procedures 

■  Status 
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Joint  Interoperability  and  Engineering  Organization 
Center  for  Information  Management 
DOD  Data  Administration  Program  Management  Office 


data  administration  program 

MANAGEMENT  OFFICE 
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for  standard  data 


DATA  ADMINISTRATION  PROGRAM 
MANAGEMENT  OFFICE 


GENERAL  VIEW  OF 

STANDARDIZATION  REVIEW  PROCESS 
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Forwarded  Issues 


COLLABORATIVE  MODELING 


MCEB  Update  25  JUL  94 


DATA  STANDARDIZATION  STATUS 


DISA\JIEO\CIM  MCEB  Update  25  JfJL  94 


DATA  ADMINISTRATION  PROGRAM 
MANAGEMENT  OFFICE 


PROGRAM  MANAGEMENT  OFFICE  (DAPMO) 
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-  related  element(s)  (o) 
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studies,  as  well  as  transcending  traditional  Earth  science  discipline  boundaries"  - 
allowing  researchers  to  readily  obtain  information  about  a  variety  of  data  sets  for 
global  change  research. 
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future  generations  of  people  on  Earth  is  a  priority. 
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following  functionality:  u;  prototype  uiciauiua  uutua^  v~, 

prototype  distributed  GCMD  database  server,  (3)  extended  WAIS  functionality,  (4) 
enhanced  functionality  of  geographic  query  capability,  recognizing  the  Content 
Standard  for  Digital  Geospatial  Metadata,  and  (5)  support  for  additional  protocols. 
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Project  Manger,  U.  S.  Global  Change  Master  Directory 
Code  902,  Global  Change  Data  Center 
NASA/Goddard  Space  Flight  Center 
Greenbeit,  MD  20771 


Introduction 

Through  the  CEOS  Working  Group  on  Daw,  represenwdves  from  the  American,  Asian,  and 
European  continents  have  collaborated  to  provide  information  about  their  countries' 
scientific  daw  sets.  This  working  group  represents  many  agencies,  universities,  and  other 
organizations  within  each  country.  The  system  of  networked  connections  among  the 
countries  that  offer  and  exchange  data  set  information  is  called  the  CEOS  International 
Directory  Network  (IDN).  Earth  science  data  set  references  presently  dominate  the 
directory.  A  recent  brochure  displays  access,  assiswnce,  and  other  services  available 
within  each  country's  domain.  The  brochures  are  available  through  the  Global  Change 
Master  Directory's  (GCMD)  User  Support  Office  -  Code  902,  NASA/Goddard  Space  Flight 
Center,  Greenbeit,  MD  20771.  One  can  also  reach  the  user  support  office  by  phone:  (301) 
441*4202  or  by  FAX:  (301)  441*9486,  or  through  the  Internee  mduso^gcmd.gsfc.nasa.gov. 
The  American  node  of  the  IDN  can  be  accessed  by  entering  NCSA's  Mosaic  and  clicking  on 
the  OPEN  button.  To  access: 

1.  The  WWW  Homepage  for  the  GCMD 
Enter  http://gcmd.gsfc.nasa.gov/intro.html 

2.  The  GCMD  Query  Form 

Enter  http://gcmd.gsfc.nasa.gov/gctnd.html 

3.  The  DIF  Writer's  Guide  on  Mosaic 

Enter  http://gcmd.gsfc.nasa.gov/difguide/difman.htinl 


Argentina  Joins  CEOS  IDN  As  Cooperating  Node 

The  CEOS  IDN  Is  an  operational  system  with  three  coordinating  nodes  representing  the 
international  science  community:  (1)  American  -  at  NASA/Goddard  Space  Flight  Center, 
Greenbeit.  MD,  USA;  (2)  Asian  •  at  the  National  Space  Development  Agency  of  Japan 
(NASDA/EGC)  in  Saitama,  Japan;  (3)  European  *  at  the  European  Space  Agency /European 
Space  Research  Institute  (ESA/ESRIN)  in  Frascati,  Italy.  These  coordinating  nodes 
mainuin  duplicate  copies  of  the  database  and  the  operational  software.  Japan  (JICST), 
Canada  (CCRS),  France  (CNES),  Germany  (DLK),  the  UK  (NRSC),  Italy  (PNRA),  countries 
represented  hy  UNEP/GRID,  and  several  agency  nodes  in  the  USA  represented  by  NQAA, 
USGS,  and  CIESIN  have  each  been  considered  "cooperating"  nodes.  These  nodes  provide  a 
path  for  researchers  within  these  countries  to  exchange  information  with  the  CEOS  IDN. 
In  March,  1994,  a  new  cooperating  node  was  installed  in  Argentina  (CONAE).  Russia, 
Australia,  Brazil,  New  Zealand,  anti  possibly  China  will  join  the  network  in  the  coming 
years.  Several  daw  set  entries  have  already  been  received  from  these  countries. 

Commitment  to  Population  of  the  Directory 

Entries  for  the  directory  are  submitted  in  a  standard  format,  the  Directory  Interchange 
Format  (DIF),  providing  standardized  information  on  parameters,  geographic  and  temporal 
coverage,  data  set  location,  and  other  summary  information  that  can  be  automatically 


loaded  into  the  database.  DIFs  are  submitted  to  the  GCMD's  discipline  "coordinators",  who 
quality-control  submitted  DIFs,  as  well  as  write  DIFs  for  important  data  sets  they 
identify.  The  value  of  the  directory  depends  on  current  data  set  information,  and  the 
effort  is  committed  to  referencing  data  sets  from  as  many  available  sources  as  possible.  In 
addition,  software  to  assist  in  formacting  data  set  entries  will  be  made  available  in  the 
coming  year. 

New  Version  of  Software  Released 

Usage  statistics  emphasize  the  increasing  popularity  of  the  directory,  with  more  than 
10,000  sessions  logged  between  January  and  May,  1994.  Directory'  users  are  offered 
several  options  for  their  interface  to  the  client-server  system,  based  on  their  terminal 
capabilities.  A  software  update  (CEOS  IDN  Installation  Package,  Version  2)  was  released 
on  June  15th,  1994,  which  incorporates  upgraded  loader  software  and  an  X  Window  client. 
Two  additional  releases  are  planned  within  the  next  year,  which  will  Include  improved 
geographic  search  capability  and  an  enhanced  X-Wlndow  client. 

The  Future 

Future  plans  include  maintaining  CEOS  IDN  node  compatibility  and  integrity,  Increasing 
data  set  population,  successfully  integrating  technological  innovations,  improving  the 
geographical  search,  enhancing  the  X- Window  client,  incorporating  the  content  standard 
for  digital  geospatial  metadata  with  the  DIF,  and  migrating  the  system  to  a  '‘distributed" 
architecture.  WAlS-based  full  text  searching  on  the  contents  of  the  directory  is  planned, 
in  addition  to  improved  keyword  searches. 
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U.S.  Representative  to  ATCCIS  [U.S.  Army  0DISC4,  OASD(C3l)-T&TC3]—  Generic  Hub  and 
Fire  Support  Data  Model  (1992-95) 

DISA/JIEO/TBCE  (Information  Directorate,  Center  for  Standards}— Fire  Suport  Data  Model 
and  C2  Core  Data  Model  (1992-93) 
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PURPOSE  OF  DATA  MODELS 
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Internal  system  exchanges  (e.g.,  FACTs) 


STANDARDIZATION  POLICY 
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Definition  based  on  data  entities  and  their  associated  attributes  established  in  the  DoD 
Data  Model 

Reflects  a  single  concept  to  promote  shareability  and  data  independence  for 
application 
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JIEO/DAPMO  INTEGRATES  C2  AND  OTHER  FUNCTIONAL  AREA 
DATA  MODELS  INTO  THE  DoD  DATA  MODEL 


DoD  Enterprise  Mode! 

STRUCTURE  OF  DoD  ENTERPRISE  MODEL 
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C2  Core  Data  Model 

CONCEPTS  UNDERLYING  C2  CORE 

DATA  MODEL 


To  describe  what  is  needed  and  what  is  allocated  to  carry  out  a  plan  to 
achieve  an  objective 
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€2  Core  Data  Model 

CONCEPTS  UNDERLYING  C2  CORE 
DATA  MODEL  (Cont'd) 
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Fire  Support  Data  Model  is  first  extension  of  the  common  core 
A  single  data  model  is  possible  (with  many  user  views) 


.  C2  CORE-INDEPENDENT  ENTITIES 
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.  C2  CORE-LOCATION  AND  TARGET 
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.  C2  CORE-ASSOCIATIONS 
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OflGANKATTON-TYPE 


.  C2  CORE-ASSOCIATION  EXAMPLE  VIEW 
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.  C2  CORE-LOCSATION  ASSOCIATION  VIEW 


Challenges 

COORDINATION  OF  C3I  MODELLING  EFFORTS 
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WORK  ON  LOCATION,  FEATURE,  METOC,  AND  COMM-ELECTRONICS 
IS  UNDERWAY;  MANY  EFFORTS  ARE  FOCUSED  ON  C2  CORE 
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-  System  specifications 
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IEEE  1320.2 
IDEF1X  Working  Group 

Update  Briefing 


What  is  IEEE  1320.2  ? 

•  Working  Group  attempting  to  define  the 
next  generation  of  IDEF1X  language 

•  Formed  from  the  group  which  wrote  the 
current  FIPS  184 

•  Mix  of  Industry  and  Government 
representatives 


Request  for  Changes  (RFC) 


•  Make  the  Formalization  more  accessible 


-  Annotated  versions  of  the  Formalization  have  been 
developed  and  are  being  reviewed 

•  Add  Constraint  Language  and  required  support 

-  Add  Support  for  Entity  Identity 

-  Make  Primary  &  Foreign  Keys  Optional 

-  Add  Names  at  either  or  both  ends  of  a  Relationship 

-  Subset  of  Language  sufficient  to  express  FK  by  1X-95 

-  Incremental  evolution  into  full  ADT  and  method 
support 


Request  for  Changes  (RFC) 

•  Allow  Discriminator  from  ancestor 

-  Allow  use  of  any  attribute  in  view  as  discriminator 

•  Support  Multiple  Inheritance 

-  Under  Discussion 

•  Re-examine  Alias  support 

-  A  meta-model  issue  that  is  being  researched 

•  Specify  the  Transform  Model 

-  Under  discussion 


Request  for  Changes  (RFC) 


•  Interchange  Format 

-  IDL  PAR  has  been  withdrawn 

-  CDIF  &  other  techniques  are  being  discussed 

•  Dictionary  Hierarchy* 

-  Item  requires  restatement 

•  Usage  Guidelines  for  Upper  ZF  Rows* 

-  Item  requires  definition 

•  Full  Method  Support 

-  Includes  support  for  ADT.  methods  with  arguments  and 
specification  language 


Other  Efforts 

•  Conformance  Guidelines 

-  NIST  -  Bruce  Rosen 

-  Less  formal  than  conformance  tests 

•  WG  Members  currently  evaluating  RCL  by 
creating  examples  in  each  of  their  areas  of 
expertise 
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Issues 

•  Lack  of  vendor  participation 

•  Funding  for  development  of  language 
components 


Next  Working  Group  Meeting 

Place:  NIST,  Building  225,  Room  B 157 

Gaithersburg,  MD 
Time:  August  7-9,  1994 

08:30pm  -  5:00pm 
POC:  Mary  Laamanen 

301-975-3260 

NOTE:  Attendees  are  expected  to  WORK! 
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Chapter  II  -  M  &  S  Environment 
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-  Terrain  (Static  and  Dynamic) 

-  Dynamic  Environment 
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•  Comp  Gen  Forces  -  TRADOC-TRAC 

•  User  Interfaces  -  AMC-STRICOM 
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-  Facilitate  application  within  development 

-  Pursue  development  of  standards  where  none  exist 

-  Work  closely  with  DOD  community 
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Objective  Action  Plans 


[  PMSO  1 


Sub-Objectives  Outline  [  PMSO  j 

•  introduction 

•  Objective 

•  Needs 

•  Vision 

•  Current  Assessments 

•  Roadmaps 

•  Recommendations/Follow-on  Issues 
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Functional  Working  Groups 


(FWGs) 


[  DMSO ; 


Tasks 

•  Validate  Needs 

•  Prioritize  the  Sub*Objectives 

•  Identify  Major  Component  Programs 

•  Participate  in  Integrated  Process  Teams 


FWGS 

•  Education.  Training  and  Military 

Operations  (ETMO) 

•  Analysis 

•  Raeaarch  &  Development  (RAD) 

•  Production  &  Logistics  (P&L) 

•  Test  dc  Evaluation  (TAE) 


MEETING  Date 
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FY95  Investment  [  DMSO  ] 


l/DBTWG 

Infrastructure  [ 

FY94 IPL  Tails 

\  wmLm 

Focused  Areas 

Build  On  Build  On 

Til  a  tael— Pin 

— 

Interoperability  Jointness  Up  Front 

•  Mission  Rthursal/  •  Terrain  •  CGF/SAFOR 
Mission  Planning 


Key  Organizations  Supporting 

Defense  Modeling  &  Simulation  [  DMSO  ^ 
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Persistence  T 

Nothing  in  the  world,  \ 

can  take  the  place  of  persistence.  \ 

S.  Talent  will  not..  \ 

\  nothing  is  more  common  than  \ 

\  unsuccessful  men  with  talent  \ 

\  Genius  will  not..  > 

\  unrewarded  genius  is  almost  a  proverb 

\  Education  will  not.. 

\  the  world  is  full  of  educated  derelicts 

\  Persistence  and  determination  alone 
\  are  omnipotent 

The  slogan  “Press  On*'  has  solved  and 
will  solve  the  problems  of  the  human  race. 

'Sad**  ffoodape 


Dr.  Patricia  Sanders 
12  July  1994 
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Mission  Statement 
Naval  Research  Laboratory 
Office  of  Strategic  Phenomena 


•  The  Offloe  of  Strategic  Phenomena  consolidates  Space  Boienoe  Divieien 
Mtivitiea  acaeeicted  with  experimental  and  modeled  data  ot  background, 
target  and  environmental  phenomena  aa  they  relate  to  the  deeign  ot 
strategic  theater,  and  tactical  military  system*. 

•  The  miaeion  ie  to  develop  and  maintain  the  meana  by  whieh  oertein 
phenomenoiegy  date  ie  to  be  archived,  distributed,  analysed  and  ueed  by  the 
community  ot  designers,  expeHmentere,  aeientiete,  and  w  erg  am  ere  working 
in  the  erase  such  ae  batliatie  mieaiie  Catenae  or  In  synthetic  envkonmente 
tor  more  comprehensive  DoO  aimulations. 

•  In  the  eaoo  of  modeled  data,  tha  miaeion  extends  to  the  means  by  whieh 
eueh  realizations  are  generated,  verified,  end  validated  tor  use  in  the  deeign, 
simulation,  and  tael  el  eurvelllaneo  sensors  and  both  defensive  end 
offensive  aye  ten*  oonoepta. 

•  The  experimentally  derived  data  pertain*  primarily  to  natural  backgrounds 
end  environments  whereas  tha  modeled  data  Includes  representations  ol 
phenomena  of  man  •made  origin,  both  ns  viewed  principally  from  apace. 


Current  Projects 
Naval  Research  Laboratory 
Office  of  Strategic  Phenomena 


Synthetic  Scan*  Generation  Model 
SSGM 


The  standard  BMD  modeling  tool  tor  generating  multi-phenomenology, 
synthetic  imagery  date  relating  to  airategie  backgrounds  end  targets 


Backgrounds  Data  Can  tar 
BDC 


The  phenomenology  data  eonter  lor  BMD  experimental  produets 
rata  ting  to  terrestrial,  atmospheric,  and  oe  lee  tie!  backgrounds 


Environmental  Effects  for 
Distributed  Interactive  Simulation 
FDIS 


The  DM 80  sponsored  effort  to  provide  the  means  to  incorporate 
sufficient  and  raaUetio  environmental  representations,  effects  & 
processes  ooneiatemty  into  Distributed  Interactive  Simulations 


Mission  &  Goals 


To  tho  extent  that  thay  impact  waapon  ayatam  parformanca  and 
attrition,  provide  tha  manna  to  incorporate  sufficient  and  raallatio 
environmental  rapraaantationa,  effecta  A  procaaaaa  eonaiatantly  in 
diatributad  interactive  aimulatkona. 


Goala:  Provide  ESP  Infraatructuro  for 


•  Senaor  Reaponee  —  recon,  surveil,  acquire ,  track,  aaaeee ... 

«  Platform  Motion  —  performance,  traffleablllty,  velocity,  acceleration ... 

*  Decision  Aida  A  Human  Factors  —  use  of  environmental  knowledge 

An  Important  Specific  Goal  1| 

“To  achieve  the  high  fld&lity  simulation  of  sensor  donation  of  targata” 

—  From  tha  DM 30  technical  evaluation  of  E*D<S  proposals 


Modeling  Issues  for 
Distributed  Interactive  Simulation 
_ (DIS) _ 


•  Adequate  modeling  of  E&E*  la  critical  to  the  raallatlc  simulation 
of  battlefield  activity  (senaing,  moving,  decision  making). 

•  Realism,  while  desirable,  is  not  an  end  unto  iiaatf  —  attrition  la 
what  counts.  It  EAEJ  doesn’t  effect  the  outcome  of  a  simulated 
battle,  it's  inclusion  in  DIS  is  difficult  to  justify. 

*  Sufficiant-tldalltv .  ohvaica-baaad  models  and  databases  are 
required  to  handle  the  variety  and  dynamics  of  the  real  world. 

*  A  broad  range  of  aealaa  and  /avals  of  fldalltv  are  required  to 
address  strategic  through  tactical  issues. 

*  The  DIS  architecture  must  handle  dynamic  anvlronmanta  in 
Interactive,  realtime  mode  for  virtual  simulation. 

•  Conalatant  raoraaantatlon  A  traatmant  of  E&E3  is  critical  for  a 
“fair-fight". 
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E2DIS 


E2DIS  Organization 


Program  Management 
4  Integration 
NSL-OC 


Teehnioel 

llanagomanl  Council 
Chair  Hfll-OC 
USN:  NPL4MV 
DtA;  MSIC 
USAS:  SUOe 
USA:  CftAIL  ♦  AMUS6 


Executive 
Adviaory  Council 
USN,  USA,  USAS 


AmhHattura 

NAk-OC 


OcmenetraUen 

njvr 


■mtramiMNai  Meofecentath 

MM-MAY 


Imkenmamai  Itteeu  4 


Summary  of  E2DIS 
System  Design  &  Development 

( Tho  Eight  Task  Areas) 


1—  Coordinate  the  Mvtn  technical  taaka 


Architecture —  Analyze,  design,  imp  lament  the  means  to  Incorporate 
E&E*  into  distributed  simulations  using  DtS  standards 

SrntY  Rwuirmtata  i  GvuMUtiu—  Identify  requirements  for  E&E* 
by  simulatora  and  define  etate-of-eclence  modeling  capabilities 

Standards—  Define  standard  database  structures,  transfer  formats, 
and  messages  to  allow  information  on  E&E3  to  be  used  in  DiS 

Environmental  fleorsasntariona—  Develop  automated  methodologies 
&  procedures  to  put  numerical  environmental  data  into  standard  form 

ElUfinmtatal  EtfKtt  A  £cacme I—  Provide  sufficient-fidelity, 
physics-based  environmental  effects  &  process  modeie  and  make  them 
available  to  simulators  using  the  E3DIS  architecture 

Demonetntlcn—  Prove  that  the  E3DIS  implementation  is  viable 

Conetruetlvn  Slmuletlona  —  Provide  the  means  to  represent  E&E3  in 
constructive  simulations 


E2DIS 


Summary  of  E2DIS 
System  Design  &  Development 

( The  Eight  Task  Areas  *  Chart  1) 


Management 
a  Intagration 


Provide  for  coordination  of  tha  aavan  taehnleal  taak 
ir»u  such  that  an  Integrated  and  proven  E*D!S 
system  and  aaaociatad  documentation  is  davolopad 
and  that  all  program  goals  are  achieved. 


A  Methodology 


Provide  a  valid  deaign  and  verifiable 
Implementation  that  allows  the  Incorporation  of 
appropriate  fidelity  phyaica  of  the  environment  and 
environmental  affects  seamlessly  into  distributed 
simulations.  The  design  A  Implementation  shall  be 
open,  reusable,  and  shall  use  current  DI8 
standards  —  or  new  ones  proposed  as  necessary. 


E2DIS 


Summary  of  E2DIS 
System  Design  &  Development 

( The  Eight  Teak  Areas  •  Chart  2) 


Survey  Requirements 
4  Capabilities 


Define  required  simulation  environments 
and  provide  basis  for  selecting  natural 
environment,  environmental  effects,  A 
environmental  process  models. 


Standards 


Facilitate  and  simplify  the  transfer  of  Information 
on  thv»  synthetic  physical  environment  used  in 
distributed  simulations  by  developing  standard 
database  structures,  transfer  formats,  prototype 
standard  databases,  and  DIS  environmental 
messages  (Protocol  Data  Units:  PDUs). 
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Summary  of  E2DIS 
System  Design  &  Development 

( The  Eight  TMk  Areas  -  Chart  3) 


Environmental 

Representations 


Facilitate  the  creation  and  representation  ot 
synthetic  environments  by  developing 
methodologies  to  treat  numerical  environmental 
databases  a  feature  models.  That  is,  to  provide 
the  means  to  put  environmental  data  Into 
standard  form  such  that  Inherent  temporal  and 
spatial  variability  is  retained  while  ensuring  that 
the  representation  Is  soaleable  to  the  constraints 
A  capabilities  of  different  simulators. 


Environmental 
Effects  &  Processes 


Select,  modify,  or  create  sufficient-fidelity, 
physics-based  environments!  affects  a 
process  models  and  to  make  them  available 
to  simulators  through  the  E*DIS  architecture. 


Summary  of  E*DIS 
System  Design  &  Development 

( The  Eight  Task  Areas  •  Chari  4) 


DemgbstiStion  |  Enable  rapid  prototyping  of  design  alternatives 

DmtnruninH  1  §PPW  10  “tl,fY  th*  E*0» 
prototyping  I  Ptan  ancJ  conduct  demonstrations  of  the 

effects  of  realistic  synthetlo  environments  on 
weapon  system  performance  to  show  the 
achievement  of  significant  E*DI8  goals  and  to 
test  a  evaluate  the  E3DIS  methodology. 


E&E*  In 
Constructive 
Simulations 


Develop  a  method  to  predict  atmospheric 
environmental  effects  on  the  performance  of 
weapons,  sensors,  and  systems  as  portrayed  in 
constructive  simulations.  The  task  will  develop  the 
FASTPROP  software  tool  to  provide  an  integrated 
capability  for  generating  a  rendering  representative 
weather  conditions  in  DtS  compatible  format  and  to 
provide  the  means  to  assess  the  Impact  of  the 
atmospheric  environment  on  signature  prediction. 


Simulation  Ocean  Terrain  Atmos  Space  Emission 

Manager  Manager  Manager  Manager  Manager  Manager 


Environmental  Services 


Managers  &  Servers 
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Verification  &  Validation 


>  Verification 

— Varittoatiofi  of  aaoti  PM  axparimant  6a panda  on  tha  davatopmant  of  a 
oonoaptuai  modal 
—Tha  Conoaptual  Modal  oontalna: 

•  AN  antxy  oanatratnte,  iaqu)rwnanta  and  tpaaMaaUana 
« amity  aahavter 

•  hrta  rate  lana  and  Interteaas  aarwaan  anittlaa  tat  oanatructlva  «va,  and  virtual  parts  at 


— Vsrifloation  wNI  bo  aooentptiahad  In  two  parts: 

a|  kaj  naaBMaOMai  Hdtla - ■»!  lidal 

■  ▼mi iif  mpmnviiiiiivii  p  wfviinini  07  Mn^pwii  win  concvpivw  iiiuumi 

•  CanMrm  that  tha  ay  rands  bahavtar  at  tha  awpartmam,  as  N  la  run,  la  as  ax  pact  ad 

Valuation 

—Validation  ot  soeh  PM  axpatinr.it  rsliss  on  staling  hypothaooo  of  a 
quantitaUva  statistical  natura  to  bo  voildatad  or  invatidatad  by  tha  raouita  of  tha 
oxpadmant 

•  POM  ax  pari  manta  ara  rtaaljntd  tataat  hypathaaaa  through  maaauramant  at  quanlKlaa 
aaaanlalad  w«h  tha  raapinaa  at  tha  OM  system  or  almutellan  antMaa  during  tha 
axpartmant  and  aaSaaauam  lampirtaan  at  thaaa  guanitltaa  vrtth  tha  atatad  hypothaaaa 

•  Analysts  ot  ROM  oxpartmsnte  la  an  Integra!  part  at  tha  axpartmantatlan  aettvtilaa  and  la 
uaad  to  vandals  hypathaaia  Aniiyate  a  alas  uaad  ic  avaluata  iha  dynamical  batuvtor 
ot  a  aanaaptual  modal, 
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E*DIS  Task  Area  2 

Standards 


•  Requirements  for  Standerd  Databases,  Date  Structures  A  Transfer  Formats 

-  Needed  to  (Militate  Irene  ter  ol  information  on  the  synthetic  phyeioei 
environments  used  in  distributed  aimuiadone 

-  Implementation  on  wide  range  of  computers  using  COTS  or  COTS  software 

-  Coordination  with  existing  DoD  efforts  at  standardization  and  submission  to  an 
appropriate  OoD  approval  prooeee  such  as  the  Corporate  Information 
Management  (C1M)  system 

-  Environmental  PDU  enumeration  tables  are  required 

•  Approach 

-  Adopt/Adapt  where  possible  prevailing  standards 

•  Deliverables 

-  Standard  transfer  formats,  definition  of  DBMS  requirements ,  survey  of 
government  database  systems 

-  Methodologies  to  put  some  E*  and  E*  Into  standard  form  to  support  Use  Casa 

-  Environmental  PDU  enumeration  methodology 


E2DIS  Task  Area  2 

Standards 

Sub- tasks 


•  Develop/ Adopt  Standard  Transfer  Format(e) 

-  CRIB,  Q ridded  Binary, 

World  Meleoroiogioal  Organization  (WMO) 

-  BUFR,  Binary  Universal  Form  for  the  Representation  of  meteorological  data, 

World  Meteoroiogioal  Organization  (WMO) 

-  HDF,  Hierarchical  Data  Format, 

National  Center  tor  8uperoomputing  Applications  (NCSA) 

-  CDF,  Common  Data  Format, 

National  Aeronautics  A  Space  Administration  (NASA) 

-  DEF,  Data  Exchange  Format, 

Communications  Interface  for  Data  Exchange  used  by  DoD  A  DoC 

-  TIFF,  Tagged  Image  File  Format, 

Servers 1  versions  tot  public  domain 

-  FITS,  Flexible  Image  Transport  System, 

International  Aatronomioal  Union  standard 

-  FIPS,  Spatial  Data  Transfer  Standard  (FIPS  173) 

•  Develop  Methodologiee  to  put  E„  A  E*  Information  into  Standard  Format 

•  Develop  Methodology  to  Provide  Environmental  PDU  Enumeration  Tables 
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E2DIS  Task  Area  3 

Environmental  Representations 


•  Requirements 

-  FaoMtate  the  creation  and  raprsao ntation  of  realistic  aynthetio  environments  (or 
t  •rrain,  oesan,  atmoaphara,  and  near  apaca 

-  Enaura  traatmant  (or  E*  raiaina  Inherent  temporal  and  spatial  variability 

-  Enaura  raprsosntation  la  aoalabla  to  oonstrainta/capabilitiso  of  different  aimulatera 

•  Approach 

-  Dovalop  mathodoiogiaa  and  prooaduraa  to  daal  with  numarioal  anvironmantal 
databases  and  (aatura  modata  (put  databaaa  or  modal  output  into  atandard  form) 

•  Dovalop  methodologies  to  oombina  outputs  (ram  anvironmantal  mod  ala  A 
databaaaa  and  (aatura  mod  ala  into  a  single  anvironmantal  representation 

-  Cooparata  with  othar  ratal  ad  projects,  aueh  aa  tha  Maatar  Environmantal  Library 
(MEL),  to  provida  oceanographic  Ea  to  broaden  tha  aoopa  at  tha  EFOIS  Uao  Caaa 

•  Deliverables 

•  Documentation  ol  anvironmantal  modal*  lor  producing  E*  databaaaa  and  moons  to 
obtain  aouraa  ooda  and/or  algorithms 

-  Soioetod  pro  to  typ  tool  (aatura  mod  ala  (or  tha  atmosphere  and  naar  apaca 

-  Mathodologiaa  to  oombina  outputs  tram  numsrio  A  (aatura  modata 

-  Initial  oat  of  aynthsdo  anvironmants  lor  tha  atmoaphara  A  naar  apoea 

-  A  DMA  oompllant  ganaral  tarrain  modal  and  a  PEGASUS  tarrain  modal  (tha  lattar  to 
support  visual,  IR,  and  RP  inpula  lor  asnaor  aimulaiore) 


E2DIS  Task  Area  3 

Environmental  Representations 

_ Sub-taaka _ 

•  Atmoapharic  Environmantal  Rapraaantationa 

-  Convart  aslaotod  existing  giiddad  Ea  databaaaa  into  atandard  tranatar  formats 

-  Craata  atmospharlo  rapraaantationa  using  aatactad  modata 

•  NORAPS  and  COAMPS  (ram  NRL-MRY 

•  WAVES,  COMBIC,  and  3TATBIC  tram  AHUBED 

•  CSSU  Iram  PLOP 

-  Identity,  uao  numarioal  A  poramatrkt  (aatura  mod  ala  tor  impravod  ra  solution 

-  Com  bins  outputs  tram  E*  databaaaa  with  hlghsr  raaolutlon  (aatura  modals 

•  Naar  Spaco  and  Uppor  Air  Environmental  Rapraaantationa 

-  Usa  tirst-prinoiplea  and  engineering  level  modata  lo  generate  databases  that 
Includs  earth's  radiation  baits,  ionosphere,  magnetosphere 

•  WBMOD,  CRRESRAD,  3D  Ray  Trace,  IAPIM  and  PIM  (ram  PUGP 

•  Tarrain  Environmantal  Rapraaantstiont 

-  Integrate  atmospharlo  objects  (tog,  dust,  amoks,  clouds)  Into  terrain 

-  Transfer  PEGASUS  database  (ram  traenputer  system  lo  SGI  ONYX 

-  Register  multi  spectral  databaaaa  to  DMA  DTED 

-  Integrate  dynamic  tarrain  modata  Into  tha  tarrain  representation  software 

-  Register  radar  Mutter  databaaa  with  tarrain  databaaa 

-  Evaluate  dlHarant  visualisation  software 


Deliverables 


•  6*018  Prototype: 

AjUImmI  AlfUaMAMlA  la  Jam  m  m  ■taaia  j 

#tT»w  vipvnmvfiw  HiipivintnpQ  in  crnh  wvw  umhimmmu  mvu 

—  Complete  documentation  ol  experimental  prcoeee  4  lessens  learned 

—  Complete  documentation  el  components,  dalab ease,  interfaces, 
eommunloatione  (*.&,  018  meeeagee,  protocole;  configuration; 
non-DtS  protocols) 

•  E*0IS  Methodology: 

—  004,000 

—  Conceptual  Design  Document 

—  Requirements  Document 

—  Interfaoe  Document 

—  Database  Description* 

•  PDIS  Tool-box: 

—  Several  Tools  (Environmental  Cell  Manager,  Environmental  Data 
Ingeator,  Object-Oriented  Potatoes*) 

—  Requirements 

—  Component  Functional  Description 

—  Component  Interface  Description 

—  Design  Dooument 

—  User  Dooument 


Status 


•  Multiple  solutions  are  being  tested  and  are  expected  to  be  viable 

•  Methodology  being  developed  is  extensible  and  robust 

•  Prototype  will  provide  tool  requirements  and  verification; 
Methodology  will  provide  tool  development 

•  Interaction  with  Maater  Environmental  Library  (MEL)  program 
will  provide  standard  data  sets 


•  40  Environmental  Databases  for  atmosphere  and  terrain  will  be 
developed  using  an  object  oriented  paradigm 


Recent  Publications 

Appearing  in  Conference  Proceedings 


0-0  Simulation  Conf.  I  I  Test  Technology  Symp. 

January  1994  I  I  March  1994 
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Office  of  Strategic  Phenomena 
Projects  tor  BMDO 


BMDO  Synthetic  Scene  Generation  Model  (SSGM):  The  standard 

BMD  modeling  tool  for  generating  multt-phenomsnology,  synthetic  Imagery 
data  relating  to  atrataglo  backgrounds  and  targets 

•  Natural  A  Man-made  Backgrounds 

—  Terrestrial  {terrain,  ocean,  ocean  Ice) 

—  Clouds 

—  Atmospheric  (emission,  absorption,  scattering) 

—  Aurora 

—  Celestial  (zodiacal,  planetary,  galactic,  a xtraga lactic) 

—  Nuclear  (background  affects  and  environments) 

•  Targets 

—  Boosting  missiles,  post-boost  vehicles,  reentry  vehicles,  satellites 
—  Target  related  phenomena  (impact  debns,  decoys,  fuel  vents) 

BMDO  Backgrounds  Data  Cantar  (BDC):  Phenomenology  data  center 

lor  BMD  experimental  products  relating  to  terrestrial,  atmospheric,  and 
so  lea  tiai  backgrounds 

•  Archive,  catalogue,  maintain,  control,  and  distribute  background  data  sate  ,* 

•  Processing  A  analysis  support  to  BMC  research  community 


BMDO  Phenomenology  Program  Structura  , 
Management  of  Experimental  &  Modeled  Data  \j} 
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IO  STANDARD  EXTRACTION  METHODOLOGY  NOR  DoD  LIBRARY 
PPLICABLE  TO  M&S  EXISTS 


DMSO  (MEL) 

Common  Data  Bases,  Real  World  Data 


-  426  - 


Consistent  from  R&D  through  to  Operations 
Historical,  Statistical,  And  4-D  Data 


DMSO  (MEL) 

Common  Data  Bases,  Real  World  Data 
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e.g.  NAVO  e.g.,  FNMOC  e.g.,  ETAC 
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Shoaling  Wave  Field  Surf  Zone 
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Department  of  Defense 
DIRECTIVE 


September  26,  1 99 1 
NUMBER  8320.1 


ASD(C3I) 


SUBJECT:  DoD  Data  Administration 


References: 


(a)  DoD  Directive  5000.11,  "Data  Elements  and  Data  Codes  Stan¬ 
dardization  Program,"  December  7,  1964  (hereby  canceled) 

(b)  DoD  5025. 1-M,  "Department  of  Defense  Directives  System  Pro¬ 
cedures,"  December  1990,  authorized  by  DoD  Directive 
5025.1,  December  23,  1988 

(c)  DoD  Instruction  5000.12,  "Data  Elements  and  Data  Codes 
Standardization  Procedures,"  April  27,  1965 

(d)  DoD  Instruction  5000.18,  "Implementation  of  Standard  Data 
'  Elements  and  Related  Features,"  March  17,  1969 

(e)  through  (r),  set  enclosure  1 


A.  REISSUANCE  AND  PURPOSE 


This  Directive: 


1.  Reissues  reference  (a). 

2.  Establishes  policies  for  DoD  data  administration. 

3.  Authorizes  the  establishment  of  and  assigns  responsibilities  for  DoD 
data  administration  to  plan,  manage,  and  regulate  data  within  the  Department 
of  Defense. 

A.  Authorizes  the  publication  of  DoD  8320. 1-M,  "DoD  Data  Administration 
Procedures,"  in  aooordanoc  with  (IAW)  reference  (b)  that  will  supersede 
references  (c)  and  (d). 

5.  Authorizes  the  establishment  of  a  DoD  Information  Resource  Dictionary 
System  (DoD  IRDS). 

B.  APPLICABILITY  AMP,  SCOBS 
This  Directive: 


1.  Applies  to  the  Office  of  the  Secretary  of  Defense  (OSD),  the  Military 
Departments  (Including  the  National  Guard  and  Reserve  components),  the  Chair¬ 
man  of  the  Joint  Chiefs  of  Staff  and  the  Joint  Staff,  the  Unified  and  Speci¬ 
fied  Combatant  Coemands,  the  Inspector  General  of  the  Department  of  Defense, 
the  Defense  Agencies,  and  the  DoD  Field  Activities  (hereafter  referred  to 
collectively  as  "the  DoD  Components"). 


2.  Applies  to  all  information  systems  (ISs)  of  the  DoD  Components, 
whether  those  systems  share  data  with  other  systems  or  not.  Hereafter,  this 


Directive  shell  use  the  general  term  "information  system  (IS)"  to  refer  to 
all  of  the  applicable  systems  and  subsystems. 

3.  Applies  to  data  in  the  ISs,  including  data  elements,  codes  and 
values,  and  symbols. 

4.  Applies  when  levying  information  reporting  requirements  I AW  DoD 
Directive  7750.5  (reference  («)). 

5.  Applies  throughout  the  life  cycle  of  the  ISs  with  management  and 
acquisition  reviews  implemented  in  DoD  Directives  7920.1  and  5000.1,  and  DoD 
Instruction  5000.2,  references  (f)  through  (h). 

6.  Applies  to  the  data  elements  and  data  values  of  systems  governed  by 
reference  (g),  including: 

a.  ISs  associated  with  office  automation;  personnel,  business,  and 
administrative  systems;  financial  accounting  and  contractual  Information  sys¬ 
tems;  and  inventory  control  associated  with  acquisition  of  programs  and  sys¬ 
tems; 


b.  Metadata  (l.e.,  data  about  data)  that  affects  system  interoper¬ 
ability  or  production  and  logistics  support. 

7.  Applies  to  data  elements  and  data  values  under  the  stewardship  (i.e., 
management  responsibility,  but  not  data  definition  ownership)  of  the  Under 
Secretary  of  Defense  (Acquisition)  Including  data  elements  and  data  values 
that  are  required  to  be  unique  to  the  operation  of  equipment  and  software 
that  are  an  integral  part  of  a  planned  acquisition  and  deployable  weapon  or 
weapons  system  and  related  test  equipment. 

8.  Does  not  apply  to  data  elements  and  data  values  that  are  required  to 
be  unique  for  use  in  cryptologic  activities,  but  does  apply  to  general 
signals  intelligence  reporting  and  the  end-products  of  cryptologic  programs 
and  systems  disseminated  to  noncryptologic  activities.  Those  cryptologic 
activities  shall  assist  in  the  development  of  any  bridging  techniques. 

C.  BEFJEITIQNS 

* 

Terms  used  in  this  Directive  are  defined  in  enclosure  2. 

d.  mm 

1.  DoD  data  administration  must  be  implemented  in  ways  that  enhance 
mission  performance  through  the  effective,  economic  acquisition  and  use  of 
data.  Two  objectives  of  DoD  data  administration  are  to: 

a.  Support  DoD  operations  and  decisionmaking  with  data  that  meets 
the  need  in  terms  of  availability,  accuracy,  timeliness,  and  quality. 

b.  Structure  the  ISs  in  ways  that  encourage  horizontal,  as  well  as 
vertical,  sharing  of  data  in  the  Department  of  Defense,  and  with  other 
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Government  agencies,  private  sector  organizations,  and  allied  nations,  con¬ 
sistent  with  national  security  and  privacy  requirements. 

2.  Data  administration  functions  include  procedures,  guidelines,  and 
methods  for  effective  data  planning,  analysis,  standards,  modeling,  configu¬ 
ration  management,  storage,  retrieval,  protection,  validation,  and  documenta¬ 
tion. 

3.  Effective  data  administration  improve,  interoperability  among  the  ISs 
and  facilitates  data  exchange,  provides  the  means  for  data  sharing,  controls  . 
redundancy,  minimizes  data  handling,  and  improves  data  integrity  by  reducing 
the  oost  and  time  required  to  transform,  translate,  or  research  the  meaning 

-of  differently  named  but  otherwise  identieal  data  elements. 

4.  Data  administration  improves  the  way  an  organization  uses  data  by  de¬ 
fining  data  structuring  rules  and  standards,  planning  for  the  efficient  use 
of  data,  and  coordinating  data  definitions  and  structures  among  organization¬ 
al  components. 

5.  The  primary  tools  of  data  administration  are  an  IRDS  and  a  functional 
data  structure  and  rules.  That  structure  and  the  rules  establish  a  framework 
within  which  to  determine  what  data  elements  should  be  standardized,  desoribe 
how  data  elements  should  be  grouped,  and  state  which  data  elements  should  be 
located  in  the  DoD  IRDS.  The  functional  data  structure  is  determined  by  the 
data  needs  of  the  organization.  The  DoD  IRDS  is  used  to  define,  structure, 
and  maintain  metadata  for  data  administration. 

6.  The  DoD  IRDS  provides  a  medium  for  defining  metadata,  cross-referen¬ 
cing,  and  consistency  checking,  and  supports  the  standardization  of  data 
element  names,  definitions,  and  relationships.  Metadata  includes  a  wide 
variety  of  data  element  information  such  as  data  element  acoess  name,  de¬ 
scriptive  name,  alternate  names,  data  element  definition,  data  type,  data 
length,  storage  format,  data  validation  rules,  and  the  functional  area  or  the 
IS  that  is  the  source  of  the  data  element. 

7.  Data  elements  are  the  fundamental  unit  of  data  used  in  the  ISs. 
Standardization  of  data  elements  will  result  in  efficiently  storing  data  in 
databases  and  files,  and  in  effectively  accessing  and  using  DoD  standard  data 
elements  by  multiple  users. 

E.  POLICY 

It  is  DoD  policy  to: 

1.  Implement  data  administration  aggressively  in  ways  that  provide 
clear,  concise,  consistent,  unambiguous,  and  easily  accessible  data  DoD-wide, 
and  that  minimize  the  cost  and  time  required  to  transform,  translate,  or 
research  differently  described,  but  otherwise  identical,  data. 

2.  Standardize  and  register  data  elements  to  meet  the  requirements  for 
data  sharing  and  interoperability  among  ISs  throughout  the  Department  of 

Defense . 
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3.  Use  applicable  Federal,  national,  and  international  standards  before 
creating  DoD  standards  or  using  common  commercial  practices. 

4.  Promote  standardization  of  data  elements  in  the  Department  of  Defense 
in  a  manner  consistent  with  requirements  for  sharing  data  among  the  OSD 
Principal  Staff  Assistants,  the  Heads  of  the  DoD  Components;  and  with  the 
other  Federal  Agencies  and  organizations  and  other  nations  under  treaty  or 
international  agreements. 

5.  Levy  the  burden  and  oost  of  conversion  to  DoD  standard  data,  regard-  , 
less  of  the  origin  of  the  requirement  for  information,  on  the  Head  of  the  DoD 
Component  responsible  for  the  DoD  IS  using  nonstandard  data,  unless  otherwise 
mutually  agreed  by  all  parties  involved,  and  the  DoD  Data. Administrator  (DoD 
DA)  is  informed  of  the  agreement. 

6.  Coordinate  applicable  standards  for  information,  information  process¬ 
ing,  and  telecommunications  I AW  DoD  data  administration  procedures. 

F.  RESPONSIBILITIES 

1.  The  Assistant  Secretary  of.  Defense  for  Command  ..Control  ._,Coimun  1  ca¬ 
tions.  and  Intelligence,  as  the  designated  senior  DoD  Information  management 
(1M)  official,  shall: 

a.  Prescribe  DoD  data  administration  policies,  procedures,  criteria, 
rules,  and  terms  for  use  by  the  Heads  of  the  DoD  Components  and  monitor  com¬ 
pliance  by  the  Heads  of  the  DoD  Components. 

b.  Issue  and  maintain  DoD  data  administration  procedures  in  coordi¬ 
nation  with  appropriate  DoD  officials. 

c.  Designate  or  assign  a  DoD  DA.  Responsibilities  of  the  DoD  DA  are 
in  enclosure  3. 

d.  Review  and  approve  the  DoD  Data  Administration  Plan  submitted  by 
the  DoD  DA. 


e.  Be  the  final  authority  for  determining  the  resolution  of  DoD  data 
administration  issues. 

f.  Represent  the  Department  of  Defense  to  other  Government  agencies, 
standards  developing  organizations,  and  industry  on  matters  pertaining  to  the 
development  and  adoption  of  data  standards  or  delegate  such  representation  to 
the  DoD  DA  or  the  appropriate  functional  data  administrator  (FDAd). 

2.  The  OSD  Principal  Staff  Assistants,  within  their  areas  of  Responsi¬ 
bility,  shall: 

a.  Represent  their  interests  to  the  Assistant  Secretary  of  Defense 
for  Command,  Control,  Communications,  and  Intelligence  and  the  DoD  DA  on  all 
matters  about  data  administration. 
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b.  Designate  a  FDAd  or  exercise  FDAd  responsibilities  for  their 
functional  categories  of  data  listed  in  enclosure  4.  Responsibilities  for  a 
FDAd  are  in  enelc  '  3. 

c.  Plan  and  provide  resources  necessary  to  effectively  carry  out 
assigned  functional  data  administration  responsibilities. 

d.  Review,  approve,  and  submit  to  the  DoD  DA  their  portion  of  the 
DoD  Data  Administration  Plan. 

* 

3.  The  Heads  of  the  DoD  Components  shall: 

a.  Designate  a  DoD  Component  DA  (CDAd)  who  shall  exercise  CDAd  re¬ 
sponsibilities,  consistent  with  section  E.,  above,  and  DoD  data  administration 
procedures.  Responsibilities  for  the  CDAd  are  in  enclosure  3. 

b.  Represent  their  interests  to  the  OSD  Principal  Staff  Assistants, 
the  DoD  DA,  and  the  FDAd  on  all  matters  for  data  administration. 

c.  Plan  and  provide  resources  necessary  to  effectively  execute  CDAd 
responsibilities. 

d.  Manage  data  IAW  section  E.,  above,  and  DoD  data  administration 
procedures. 

C.  PROCEDURES 

1.  The  DoD  data  administration  procedures  shall: 

a.  Implement  the  policy  in  section  E.,  above. 

b.  Define  and  implement  strategies  and  criteria  for  converting  from 
nonstandard  data  elements  to  DoD  standard  data  elements. 

c.  Develop  requirements  for  methods  and  capabilities  that  permit 
rapid  generation  and  manipulation  of  data  models. 

d.  Apply  to  all  data  elements  that  are  used  in*,  but  are  not  limited 
to,  the  functional  areas  listed  in  enclosure  4. 

2.  DoD  data  administration  procedures  shall  provide  uniform  instructions 
for  implementing  DoD  data  administration.  These  procedures  shall: 

a.  Identify  planning,  reporting,  and  resources  requirements  for 
effective  DoD  data  administration. 

b.  Establish  DoD  standard  data  element  naming  conventions  and  uni¬ 
form  procedures  to  define  and  maintain  all  DoD  standard  data  elements. 

c.  Describe  the  means  to  satisfy  all  data  requirements  for  new  or 
modified  ISs  through  the  use  of  standard  data  elements. 
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d.  Describe  the  detailed  administrative  relationships  among  the  DoD 
DA,  the  FDAds,  the  CD  Ads,  and  the  users  of  data. 

e.  Provide  guidance  for  IRDS  users,  including  how  to  access  and  use 

the  metadata. 

f.  Identify  the  mechanism  to  structure,  store,  collect,  and  maintain 
metadata  within  the  Department  of  Defense  so  that  metadata: 

(1)  Is  readily  accessible  to  and  understood  by  the  Heads  of  the  . 

DoD  Components. 

(2)  Can  be  made  available  to  commercial  enterprises  proposing  or 
developing  defense  systems. 

(3)  Is  protected  1AW  the  FAR  and  DoD  Directive  5200.1  (refer¬ 
ences  (i)  and  (J)).  | 

g.  Determine  the  relationships  and  applicability  of  DoD  data  admin¬ 
istration  to  references  (e)  through  (h)  and  (k)  through  (o)  that  govern  the 
ISs  of  the  Heads  of  the  DoD  Components. 

3.  DoD  standard  data  elements  shall  be  used  when  stating  information 
requirements  and  when  designing,  developing,  or  modifying  the  ISs.  Com¬ 
pliance  shall  be  determined  by  officials  authorized  to  review  and  approve 
information  systems  IAU  DoD  Directives  7750.5,  7920.1,  and  5000.1  (references 
(e)  through  (g)). 

4.  Nonstandard  data  acquired  from  oonnercial-off-the-shelf  data  sources 
or  other  sources  external  to  the  Department  of  Defense  shall  be  converted  to 
DoD  standard  data  elements  only  when  justified  by  mission  requirements,  fea¬ 
sibility  analysis,  and  a  cost-benefits  analysis. 

H.  EFFECTIVE  -MTE 

This  Directive  is  effective  immediately. 

<~Z5ZJu 

Donald  J.  Atwood 

Deputy  Secretary  of  Defense 
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DEFINITIONS 

1.  Automated  Information  System  (AIS).  A  combination  of  information,  com¬ 
puter,  and  telecommunications  resources  and  other  information  technology  that 
collects,  records,  processes,  stores,  communicates,  retrieves,  and  displays 
data. 

2.  Data.  Representation  of  facts,  concepts,  or  instructions  in  a  formalized 
Banner  suitable  for  communication,  interpretation,  or  processing  by  humans  or 
by  automatic  means.  Any  representations  such  as  characters  or  analog  quanti¬ 
ties  to  which  meaning  is,  or  might  be,  assigned  (Joint  Pub  1-02  (reference 

(P»>. 

3.  Data  Administration.  The  responsibility  for  definition,  organization, 
supervision,  and  protection  of  data  within  an  enterprise  or  organization  (NBS 
Special  Publication  500-152  (reference  (q))). 

M.  pat a  Administrator  (DA).  A  person  or  group  that  ensures  the  utility  of 
data  used  within  an  organization  by  defining  data  policies  and  standards, 
planning  for  the  efficient  use  of  data,  coordinating  data  structures  among 
organizational  components,  performing  logical  data  base  designs,  and  defining 
data  security  procedures  (reference  (q)). 

5.  Database.  A  collection  of  interrelated  data,  often  with  controlled 
redundancy,  organized  according  to  a  schema  to  serve  one  or  more  applica¬ 
tions;  the  data  are  stored  so  that  they  can  be  used  by  different  programs 
without  concern  for  the  data  structure  or  organization  (ANSDIS,  ANSI/X3.172- 
1990  (reference  (r))). 

6.  Database  Administrator.  A  person  or  group  that  enforces  policy  on  "how." 
"where,"  and  "in  what  manner"  data  is  stored  and  maintained  in  eaoh  database. 
Provides  information  to  the  DA  on  organizational  use  of  data  within  the  sub¬ 
ject  database. 

7.  Data  Dictionary.  A  specialized  type  of  database  containing  metadata  that 
is  managed  by  a  data  dictionary  system;  a  repository  of  information  describ¬ 
ing  the  characteristics  of  data  used  to  design,  monitor,  document,  protect, 
and  control  data  in  ISs  and  databases;  an  application  of  a  data  dictionary 
system  (reference  (q)). 

8.  Data  Element.  A  basic  unit  of  information  having  a  meaning  and  subcate¬ 
gories  (data  items)  of  distinct  units  and  values  (reference  (p)). 

9.  Data  Item.  A  subunit  of  descriptive  information  or  value  classified 
under  a  data  element  (reference  (p)). 

10.  Data  Model.  Identifies  the  data,  their  attributes,  and  relationships  or' 
associations  with  other  data. 

11.  Data  Value.  A  value  associated  with  a  data  element.  One  of  the  allowa¬ 
ble  values  of  a  data  element.  Synonym  of  "a  data  item." 

12.  Functional  Area.  A  range  of  subject  matter  grouped  under  a  single  head¬ 
ing  because  of  its  similarity  in  use  or  genesis. 

‘3.  : 'nagerv .  Collectively,  the  representations  of  objects  reproduced  elec¬ 

tronically  or  by  optical  means  on  film,  electronic  display  devices,  or  other 
media  (reference  ( p ) ) . 
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14.  Information.  The  meaning  that  a  human  assigns  to  data  by  means  of  the 
known  conventions  used  in  their  representation  (reference  (p)). 

15.  Information  Resource  Dictionary  System  (IRDS).  A  set  of  standard 
specifications  for  a  data  dictionary  system  resulting  from  U.S.  Federal  and 
national  standards  efforts;  a  computer  software  system  conforming  to  those 
standards  that  provides  facilities  for  recording,  storing,  and  processing 
descriptions  of  an  organization's  significant  information  and  information 
processing  resources  (NBS  Special  Publication  500-152  (reference  (q))). 

16.  Information  System  (IS).  A  combination  of  information,  information 
technology,  and  personnel  resources  that  collects,  records,  processes, 
stores,  communicates,  retrieves,  and  displays  either  manually  or  with  varying 

-degrees  of  automation. 

17.  Metadata.  Information  describing  the  characteristics  of  data;  data  or 
information  about  data;  and  descriptive  information  about  an  organization's 
data,  data  activities,  systems,  and  holdings  (reference  (q)). 

18.  Nonautomated .  Manual,  without  benefit  or  hindrance  of  machines. 

19.  OSD  Principal  Staff  Assistants.  The  Under  Secretaries  of  Defense,  the 
Assistant  Secretaries  of  Defense,  the  General  Counsel  of  the  Department  of 
Defense,  the  Inspector  General  of  the  Department  of  Defense,  the  Comptroller 
of  the  Department  of  Defense,  the  Assistants  to  the  Secretary  of  Defense,  the 
OSD  Directors  who  report  directly  to  the  Secretary  or  Deputy  Secretary  of 
Defense,  and  the  DoD  Coordinator  for  Drug  Enforcement  Policy  and  Support. 

20.  Signals  Intelligence  (SIGINT).  A  category  of  intelligence  information, 
either  individually  or  in  combination,  comprising  all  communications  intelli¬ 
gence,  electronic  intelligence,  foreign  instrumentation  signals  intelligence, 
and  telemetry  intelligence  (reference  (p)). 

21.  Standard.  An  exact  value,  a  physical  entity,  or  an  abstract  concept 
established  and  defined  by  authority,  custom,  or  common  consent  to  serve  as  a 
reference,  model,  or  rule  in  measuring  quantities  or  qualities,  establishing 
practices  or  procedures,  or  evaluating  results.  A  fixed  quantity  or  quality 
(reference  (p)). 

22.  Standard  Data  Element.  Data  element  registered  IAW  DoD  data  administra¬ 
tion  procedures. 

23.  Symbology.  Any  graphic  representation  of  concepts’or  physical  objects. 
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POD  DATA  ADMINISTRATORS’  RESPONSIBILITIES 


A.  The  DoD  Data  Administrator  shall: 

1.  Implement  and  manage  the  DoD  data  administration  policies  and  proce¬ 
dures  IAW  section  E.,  above,  of  this  Directive  and  DoD  data  administration 
procedures. 

2.  Interpret  DoD  data  administration  policies,  procedures,  and  stan¬ 
dards,  and  coordinate  FDAd  and  CDAd  procedures  with  the  DoD  data  administra¬ 
tion  procedures. 

m 

3.  Register  metadata  from  functional  areas  assigned  as  DoD  standard 
data. 

4.  Develop,  operate,  and  maintain  a  DoD  IRDS  that  is  easily  accessible 
to  all  the  Heads  of  the  DoD  Components  and  users,  and  supports  the  DoD  data 
administration  procedures . 

5.  Review  and  approve  or  disapprove  proposed  changes  to  DoD  standard 
data  elements,  when  existing  standard  data  elements  do  not  satisfy  new 
requirements . 

6.  Annually  review  and  submit  the  consolidated  and  updated  DoD  Data  Ad¬ 
ministration  Plan  to  the  DoD  Senior  IM  official  IAW  DoD  data  administration 
procedures ,  and  DoD  4120.3-M  (reference  (o)). 

7.  Plan  and  provide  resources  necessary  to  effectively  carry  out  DoD 
data  administration  responsibilities,  giving  consideration  not  to  abridge  the 
authority  and  responsibility  of  the  OSD  Principal  Staff  Assistants. 

B.  A  Functional  Data  Administrator  shall: 

1.  Implement  data  administration  procedures,  IAW  section  D.,  above,  of 
this  Directive  and  DoD  data  administration  procedures,  for  the  functional  area 
assigned. 

■ 

2.  Approve  metadata  in  the  respective  functional  areas  of  responsibility 
listed  in  enclosure  4,  and  for  proposed  DoD  standard  data  elements,  only  when 
an  existing  standard  does  not  support  the  new  requirement. 

3.  Annually  review,  update,  and  prepare  the  portion  of  the  DoD  Data 
Administration  Plan  that  addresses  the  functional  area  assigned,  and  submit 
to  the  DoD  DA,  through  the  OSD  Principal  Staff  Assistants. 

* 

4.  Recoanend  functional  data  elements  for  standardization. 

C.  A  Component  Data  Administrator  shall: 

1.  Manage  the  DoD  Component  Data  Administration  IAW  section  E.,  above, 
of  this  Directive  and  the  DoD  data  administration  procedures. 
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2.  Review  proposed  changes  to  DoD  standard  data  elements  and  forward 
changes  to  the  DoD  DA  and  the  appropriate  FDAd  for  approval. 

3.  Identify  the  interface  between  the  users,  database  administrators, 
and  application* developers  of  the  ISs  within  the  DoD  Component  and  act  as  the 
liaison  to  the  DoD  DA  and  the  FDAds. 

Represent  CDAd  interests  to  the  OSD  Principal  Staff  Assistants,  the 
DoD  DA,  and  the  FDAds. 

5.  Annually  review,  update,  and  submit  to  the  DoD  DA  the  portion  of  the 
DoD  Data  Administration  Plan  that  addresses  DoD  Component  data 
administration. 


OSD  PRINCIPAL  STAFF  ASSISTANT 
FUNCTIONAL  AREAS  OF  RESPONSIBILITY 


This  listing  is  descriptive,  not  mandatory.  See  appropriate  charter 
directives. 


k.  The  Under  Secretary  of  .Defense  (Acquisition) 

-  Acquisition  -  Nuclear,  atomic  energy 

-  Research  and  engineering  -  Basic,  applied  research 

•  Science  and  technology  -  Development,  test,  and  evaluation 

-  Modeling  and  Simulation  -  Weapon  System  Interoperability 

•  Weapon  and  Weapon  System  Support  Engineering,  Design  and  Test 

B.  The  Assistant  Secretary  of Defense  (Production  and  Logistics) 


Logistics 
Transportation 
Procurement,  contracts 
Construction 

Real  property  acquisition, 
repair,  use,  and  disposal 


-  Configuration  management 

•  Reliability,  maintainability 

-  Manufacturing ,  materiel 

-  Base  operations 

•  Standardization 


C.  Ihe_.  Under.  Secretary  of  Defense  for  Policy 

-  National  security  -  Trade  security 

-  Strategic  resources  -  Civil  defense 

-  Environment  preservation  -  Crisis  management 

-  International  security 


D. 


Ihe-ftaalstant. Secretary  of -Defense  for  Command.  Control. 
Communications,  and  Intelligence 


-  Intelligence,  imagery 

-  Telecommunications 

-  C3i  systems 

-  Investigative  security 

-  Information  resources 

management 

-  Country  and  political  Jurisdiction 


Mapping,  charting,  and  geodesy 

Audio,  visual 

Defense  security 

Information  management 

Sea/air/ground  operations, 
fire  support 

codes 


M-1 
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E.  The  Comptroller  of  the  Department  of  Defense 

-  Budget  -  Fiscal 

-  Finance  -  Accounting 


F.  The  Assistant  Secretary  of  Defense  (Force  Management  and  Personnel) 

-  Civilian  personnel  -  Military  personnel,  manpower 

-  Military  dependents  -  Unit  administration 

-  Mobilization  -  Compensation 

-  Training  and  education  -  Equal  employment  opportunity 


G.  The. Assistant  Secretary,  of  Defense  (Health  Affairs) 

-  Health,  medical  programs  -  Health,  medical  care 

•  Military  dependent  health  affairs 


H.  The . Assistapt_Secretarv  of  Defense  (Legislative  Affairs) 
-  House  affairs  -  Senate  affairs 

•  Legislation  -  Liaison 


I.  Thg-Aasistant_Secretarv  of  Defense  (.Public  Affairs) 

-  Public  communication  -  Community  relations 

-  Dissemination  of  information  -  Freedom  of  Information 

-  Defense  news,  public  information  activities 

J.  The.. Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 

-  Defense  program  analysis  -  Defense  program  evaluation 

-  Economic,  resource  planning  -  Theater  assessment,  planning 

K.  Ibt  Assistant  .Secretary  of  Defense  (Reserve  Affairs) 

~  Reserve  personnel  -  Reserve  military  dependents 

-  Military  technicians  -  Reserve  compensation 

l.  Ihg-Aaslafeant-Secretary  of  Defense  (Special  Operations /Low  Intensity 
Conflicts) 

-  Special  operations/low  intensity  conflicts  (SO/LIC)  mission  assessment 

-  SO/LIC  requirements,  planning  -  Terrorism 


14-2 


-  Audit  -  Criminal  investigations 

-  Inspections 

The  General  Counsel  of  the  Department  of  Defense 

-  Legal,  regulation  -  Standards  of  conduct 


Defense 

•  Privacy  Act  -  Organization,  management  planning 

-  DoD  history 


BATTLE  FORCE  TACTICAL  TRAINING  (BFTT) 
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BATTLE  FORCE  TACTICAL  TRAINING  (BFTT) 


ACCELERATE  TO  SUPPORT  ATO/TTS  AND  DOWNSIZING 


BATTLE  FORCE  TACTICAL  TRAINING  (BFTT) 
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BFTT  Engineering  Organization 
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>■  Test  and  evaluation,  research  and  development 
Doctrine  development  and  evaluation 
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BFTT  SHIP  CLASSES  AND  SHORE  NODES 
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CM*  ONLY 


BFTT  OVERVIEW:  TEAM  TRAINING  MODEL 


System  Update 
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Development  Test  —  IIB  (DT-HB) 
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BFTT  System  Update 


M&S/ADS  Technology  implementation 
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>*  Coordinates  with  DMSO  and  other  DoD 
database  users  to  define  standard  database 
architecture,  models,  and  DIS  enumeration 


Applied  Research  Laboratories 
THE  UNIVERSITY  OF  TEXAS  AT  AUSTIN 


Support  legacy  databases 


Applied  Research  Laboratories 
THE  UNIVERSITY  OF  TEXAS  AT  AUSTIN 


-  479  - 


2  o  u) 

3  *5.  o  c 

i  e  %  =5 

co  »  S  m  -a 

v  >.  Z  8  £ 

>  -a  £  g  5 

o  S'  §£  g 

ffi  z  S  w  O 

H  !t{ 
|  «  8?  I 

m  S  N  *  S 
■8  e  —  c 

3  .2  =  s  Z 

£  £  E  t  75 

1  3  £8  § 


5  A  A 


E 

LU 

Q 

II 

CO  CO 

EO 

5  uj 
.£9 

So 

JC  o 

go 

S  > 

lm  JQ 

g-XJ 

m  0> 
CO  4^ 

*  CO 
GO  o 

i? 

5  « 

CO  J? 

00  .E 

© 

<0TJ 

a.  o 
E  £ 


7  a> 

^  co 

>  5 

Xi  xt 

■d  a 

0)  CO 

co  a 

3  co  E 

o®  3 

§‘i  E 

O  CO  5 

cL*5  3- 

|o  O 

|8  £ 

C  ‘C  co 

w  G)  O 

C/>  CO  O 


£S 

D)“ 

•MS  ta 

’DT 
CO  Q 

2a 

»« 

fl 

•So 

oec 

i  ^ 

o  o 

O  CO 
5*0) 

o  — 

§  c 

$  CO 

CD  C 
XI  0) 

O  «- 

o  a> 

C  *»  '**•' 
CO  3  CD 

M  44  (0 

CO  3  CO 

m 


-  480  - 


PROPOSED  BFTT  POAAM  -  05  JULY  10*4 
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APPENDIX  B:  COMPLEX  DATA  TASK  FORCE 
MEETING  BRIEFING  CHARTS 


t 


Organization  of  Complex  Data  Task  Force 


Complex  Data  Pilot  Studies  Categorization  Taxon  mies 

—  Coordinated  and  Guidelines  —  Ins  Kameny  (co-chair 

by  Chien  Huo  —  Len  Seligman  (co-chair)  —  Dan  Hogg  (co-chair) 

—  Peter  Valentine  (co-chair) 


Eighth  Modeling  and  Simulation 
l/DB  Task  Group  Meeting 
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Complex  Data  Modeling  Workshop  for  Modeling  and  Simulation  (MAS),  (Draft), 
Center  for  Standards,  M&S  Working  Paper  1-2, 14  September,  1993 


Goals  of  the  Complex  Data  Task  Force 
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October  1993:  Complex  Data  Meeting 
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November  1993:  Complex  Data  Working  Group 
at  MORS  SIMDAT  Mini-Symposium 
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February  1994:  Complex  Data  Task  Force  Meeting 
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Coupled:  e.g.f  Sex-Marital-Code  bundles  information  on  a  person's  Sex 
and  Marital-Status  in  a  single  data  element 


Items  of  Interest  Since  April  1994  Meeting 


-  494  - 


3 

0  *0 
—  (1) 

2>* 
—  03 
O  0) 

5*  73 
3  C 
•n  « 

1  i 

C  CO 

3  « 

3  £ 


cs  a) 
*o  -O 
c  — 
re  re 

4-»  "O 

«  O 

eg  E 


2  « 

£  re 

C  h- 

o  re 

■  N 

4^  - 

re  sr 

+->  *> 

3  +* 
o.  re 

E  ts 

O  qJ 

O  £ 

O  0 
13  3 
’3  __ 

C  (Mn 

.2  $ 

w  _i 

(0  ^ 

C  _J 
•“  -J 

re  4- 

c  re  S 
a  £0 
.2  a  <o 

re  =  T" 
■S  2  CD 
a)  C3  t- 

<o  _  4- 

0  O)  0 
2.  C  3 

Q-  •—  a> 

0  0)  3 

GC  TS 
ra  °  . 

Q  E  re 
c  re  a 

o  CO 
o_  >»  E 
o«  2 
3  re  c 
w  t;  re 
re  </) 

u  £  re 
o  .5  re 

&  o  51 


(0 

re  t 
re  o 

-  Jr  IL 

(o  re 

C  c  • 

*3  re  In 
re  2  ■» 

re  —  — 
i=  5  re 

co 

3  5  o 
re  —  £ 

2  *o  ° 
.2  o  re 
»-  2  co 
5  J  c 

-  re  «) 

x  co 

re  o  re 

q_  a  “*■ 

c  o  5 

°  a  o 

°  o>  2 

re  .E  T3 

|*£ 

fc  <D  3 

cn  2  cr 

c  re  re 

E  —  c 
o  «  § 
re  3 
re  2  re 

•a  E  E 

re  c 
s  o 
re  o  ^ 

2  re  *“ 

re  E  .2 

^1—4-4 

o  o  2 

E**s  re 

.2  E 

2  v  0 
:•=  re  w 


o  Q 

<0  re 
o  :t: 

■*=  c 
re  0 

■o  o 

C  CO 

re  o 

<0  c 
.3  .= 
C  -a 

3  0 

"  IS 

tg 

«  g. 

1  § 

0  2 

0).= 

co  w 
c  .E 

O  0 
'3  3 

2  >* 

c  v. 
0  c 

0  0 

re  fc 

0  re 
*“  0 
2  £ 


o  c  c 
«  g  g 

re  ^  g 
<  a  E 

s  w  re 


E  re>  0 
o  0  .2 

=  03 
o  Dll 
3=3 
c  2  re 

S  -  T3 

E  3  c 
0  o  re 

w  0  re 
o  -  re 

•s  2  -S 
c  a  p 
3  c  E 
o  E  re 

Ere  k- 
x  re 
re  0  a 


«  2  re 
re  *3  co 

=  3-0 

rer -re 

■o  g.  § 

o  a  *#r 

4->  tr  JO 
.2  0  O 

as  2 

2  3  c 

■5  re  o 

J2  3  '•= 

ore" 

5  £  re 

*-  TT  3 

°  &  .2 
0  re  > 
0  >  •. 

o  >  <n 
a  c  ■£ 

3  re  ® 

a*-  £ 

•  re  zz 
:  tj  5 
0  ~  O 

44  o  k 

c  3  *> 
2  -3  c 
E  c  0 
*2  0 
C  44 

O  O  c 

.is  0  re 

>  a>  E 


1 1 

!•! 
0  >- 
> 

■g  ^ 

®  s 

w  ■“ 

o  C7) 
o  re 

4- 

to  re 

r-  u- 

C  4-< 

re  0 


0 

co 

m 
a  s 

o  re 
o  0 

0  2 
o  u 
re  £ 

4-*  re 
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Objects  Are  in  the  Eye  of  the  Beholder 

•  Object  models  can  be  used  at  the  enterprise  level,  as  well 
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Object  Orientation  at  the  Enterprise  Level 
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Object  Orientation  at  the  Architecture/ 
Technology  Level 
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Object  Orientation  at  the  Analysis  and 
Logical  Design  Level 
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-  How  does  the  logical  design  map  to  screens, 
dialogues,  and  transactions? 

•  Some  guidance  in  layering  provided  by  HOOD 


Management  and  Organizational  Impacts 
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•  Managing  the  shifts  in  user  expectations 

-  Best  suited  to  interactive  design  and  development. 
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Definition  based  on  data  entities  and  their  associated  attributes  established  in  the  DoD 
Data  Model 

Reflects  a  single  concept  to  promote  shareability  and  data  independence  for 
application 
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.  C2  CORE-ASSOCIATION  EXAMPLE  VIEW 
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DoD  Enterprise  Mode! 

ROLE  OF  DoD  ENTERPRISE  MODEL 

PROVIDES  A  DEFENSE  ENTERPRISE-LEVEL  VIEW  OF  THE  DoD 
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JIEO/DAPMO  INTEGRATES  C2  AND  OTHER  FUNCTIONAL  AREA 
DATA  MODELS  INTO  THE  DoD  DATA  MODEL 
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-  A  single  data  model  is  possible  (with  many  user  views) 
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FOCUS  ON  JOINT  AND  COMBINED  INFORMATION  EXCHANGES  BASED 
ON  NEED  LINES  IDENTIFIED  iN  THE  FIAs 


C2  Subfunctional  Area  Data  Models 

C2  SUBFUNCTIONAL  DATA  MODELLING 
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WORK  ON  LOCATION,  FEATURE,  METOC,  AND  COMM-ELECTRON1CS 
IS  UNDERWAY;  MANY  EFFORTS  ARE  FOCUSED  ON  C2  CORE 
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STANDARDS  FOR  DOCUMENTATION  AND  ANALYSIS 

DOCUMENTATION 

High-level  IDEFO  activity  model  with  diagram  and  definitions 
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-  System  specifications 
Integration  to  DoD  Enterprise  Model 

Relation  to  DoD  policy,  guidelines,  and  architectures  (e.g.,  FIAs) 


Challenges 

APPLICABILITY  TO  DATA  STANDARDIZATION 

ACTIVITIES 
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Non-message-based  exchanges 


Challenges 

CAUTION  ON  APPLICABILITY 
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RPW  6-20-94-23  UNCLASSIFIED 


CANDIDATE  ELEMENTS  OF  A  C2  DATA  MIGRATION  STRATEGY 
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APPENDIX  C:  DATA  STANDARDS  TASK  FORCE 
MEETING  BRIEFING  CHARTS 


fS  Need  for  DoD  Data  Ak  .  "DIS  Need  for  DoD 

Standards  Paper"  UK?  Data  Standards" 


MAJ  Walt  Swindell.  TRAC 
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Concept  Brief 

&S  Distributed  Information  Resource  Repository  (IRR)  System 
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M&S:  preparing  inputs,  executing  M&S,  post-processing,  planning 
and  executing  exercises  and  experiments 


Information  Resource  Repository 
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MODELING  AND  SIMULATION  FOR  DOD  MODELING 
APPLICATIONS  IN  SUPPORT  OF  THE  WARFIGHTER 

MISSION  DESCRIPTION 


I.  BACKGROUND  AND  INTRODUCTION 

Modeling  and  Simulation  provide  a  way  to  analyze  very  complex  processes  that 
consist  of  many  interacting  components  [1].  Models  and  simulations  provide  an 
integrated  basis  for  analysis  and  decision  making  [1].  Models  and  simulations  also 
provides  a  way  to  extrapolate  existing  processes  and  workloads  to  new  situations 
[1].  Generally,  Modeling  and  Simulations  realistically  portray  complex 
operations  and  interactions  that  allow  the  users  to: 

1.  Clarify  thinking 

2.  Select  an  altemative(s) 

3.  Make  decisions 

4.  Test  hypotheses 

5.  Plan 

6.  Train  others 

Modeling  and  Simulation  generally  falls  into  one  of  two  classes: 

1 .  Exploratory 

2.  Consolidated 

Exploratory  models  are  generally  used  to  better  understand  the  hypotheses  or 
theory  regarding  some  system  when  there  are  not  hard  facts  known  about  the 
situation  being  modeled.  Modelers  must  make  guesses  about  the  facts  regarding 
the  modeled  situation.  The  executing  model  then  behaves  as  if  the  guesses  made 
about  it  were  correct. 

Consolidated  models,  in  contrast  to  exploratory  models,  model  situations  in  which 
there  is  a  very  high  level  of  confidence  about  the  facts.  These  facts  are 
commonly  consolidated  into  automated  packages,  e.g  a  COTS  spreadsheet,  which 
is  then  used  as  a  surrogate  for  the  situation  being  modeled.  These  models  behave 
very  much  like  the  real  situation  and  for  that  reason  may  be  used  to  predict 
system  behavior. 
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Models  vary  in  size  from  small  spreadsheets  to  very  large  simulations  that 
require  multiple  computer  resources  existing  in  different  geographic  locales  with 
several  hundred  people  involved  and  several  months  to  produce  output. 

The  material  below  presumes  a  working  knowledge  of  Enterprise  Database 
models  and  techniques  as  contained  in  the  text  Enterprise  Database  in  a 
Client/Server  Environment.  The  model  described  below  is  the  mission  model 
( Chapter  5 ). 

Section  II  below  contains  the  top  level  of  the  models  and  simulations  mission 
description.  This  description  is  intended  to  define  the  outside  boundaries  of  ail  the 
missions  appropriate  for  the  successful  and  quality  accomplishment  of  models  and 
simulations.  NOT  stated  or  even  to  be  inferred  is  how  these  missions  are 
accomplished,  nor  who  accomplishes  them.  These  are  not  stated  because  missions 
are  independent  from  organization  and  procedure. 

After  the  top  level  of  the  mission  is  “agreed  to”  creation  of  two  or  more  lower 
levels  will  begin.  At  the  end  of  the  effort  (probably  two  weeks)  from  the  date  of 
this  document,  the  set  of  all  models  and  simulations  missions  will  be  complete. 
Thereafter,  each  lowest  level  node  on  the  hierarchies  will  be  examined  to 
detennine  the  data  domain  associated  with  that  node. 

An  entity  relationship  diagram  will  be  created  for  each  node.  All  ERDs  will  be 
semantically  merged  to  ensure  entity  non  redundancy.  That  ERD  will  then  be 
declared  the  data-requirements-scope  for  the  Modeling  and  Simulation 
repository.  It  will  have  to  be  reviewed  and  when  (not  if)  changed,  adjustments  to 
this  mission  statement  must  take  place. 

Once  the  ERD  is  reviewed  and  approved,  it  will  be  fully  attributed  and 
transformed  into  third  normal  form.  The  business  primary  keys  will  be 
identified,  and  the  referential  integrity  &  actions  determined.  Upon  review  and 
acceptance  that  will  complete  the  activities  associated  with  determining  the 
metadata  requirements  of  the  Modeling  and  Simulation  Repository. 

The  material  below  describes  the  mission  of  Modeling  and  Simulation.  These 
mission  statements  should  not  be  viewed  as  identifying  required  agencies/sub¬ 
agencies  within  a  Modeling  and  Simulation  environment/community.  In  addition, 
not  all  the  submissions  identified  below  are  required  for  each  and  every  Model 
and  Simulation.  For  example,  Model  Dispersion  and  Support  is  a  sub-mission  of 
the  overall  Modeling  and  Simulation  mission.  Some  models  may  not  be  dispersed 
nor  supported;  thus,  the  components  of  this  sub-mission  are  not  necessary  to 
successfully  support  the  overall  mission.  Many  times  models  are  constructed  and 
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not  documented  since  documentation  can  double  the  cost  of  developing  a  model. 
The  point  is  that  not  every  model  requires  all  the  mission  components. 


II.  HIGH  LEVEL-MODELS  AND  SIMULATIONS  MISSION 
DESCRIPTION 

The  successful  accomplishment  of  models  and  simulations  requires:  model 
development;  project  management;  validation,  verification  and  certification;  data 
collection,  model  dispersion  and  support,  model  operation,  and  scenario 
generation. 

1 .  Model  development  results  in  the  development  of  requirements,  designs, 
implementations,  tests,  and  maintenance  of  the  model 

2.  Project  Management  includes  pla/s,  organization,  direction,  controls, 
monitoring,  and  staff  to  insure  that  Models  and  Simulations  are  developed 
on  time,  on  budget,  and  meet  requirements. 

3 .  Validation.  Verification,  and  Certification  ensures  that  the  model  is  a 
correct  representation  of  the  situation,  that  it  generates  correct  repeatable 
output,  and  that  some  agency  formally  declares  the  model  suitable  for 
specified  usage  in  a  specified  manner. 

4.  Data  Collection  provides  model  developers  with  information  used  to 
construct  a  model  that  fairly  represents  the  situation.  This  data  provides 
insight  into  the  range  or  intervals  where  use  of  the  model  is  indicated, 
support  for  scenario  generation,  support  for  requirements  definition,  and 
level  of  model  aggregation.  Data  collection  is  also  required  to  provide 
input  for  model  operation.  This  data  may  be  contained  in  a  data 
dictionary  or  Data  Base  and  can  be  discovered  by  surveying  the 
literature,  browsing  repositories,  interviews,  and  empirical  data 
acquisition. 

5.  Model  Dispersion  and  Support  refers  to  training  (plans,  manuals, 
procedures),  support  groups,  data  sharing,  and  shipping  that  facilitate  a 
widf*  range  of  model  use. 

6.  Model  Operation  refers  to  activities  required  to  manage  an  executing 
model.  Controllers  are  required  to  make  certain  that  all  model 
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components  are  synchronized,  monitors  collect  output,  coordinators 
ensure  that  all  needs  are  being  serviced,  model  operators  or  participants 
are  required  to  make  the  model  react. 

Scenario  Generation  refers  to  scripting  model  runs.  A  scenario  specifies 
the  operational  environment,  e.g.  in  a  flight  simulator  ,  whether  the 
landings  are  during  the  day  or  night,  if  the  landings  are  aboard  ship  or  on 
land  etc..  A  scenario  also  specifies  the  system  configurations,  i.e.  the 
values  the  various  factors  that  the  model  needs  to  produce  output  (input 
parameters  ,  structural  assumptions)  may  assume.  Scenario  generations 
are  the  result  of  an  "Experiment  Design  Plan". 

Output/Analvsis  focuses  on  the  model  output.  For  consolidated  models 
the  output  is  the  answer  and  no  analysis  is  required.  For  exploratory 
models  the  output  is  analyzed  to  ascertain  model  accuracy,  refine  the 
model,  test  hypotheses  etc.. 
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III.  DETAILED-MODELS  AND  SIMULATIONS  MISSION 
DESCRIPTION 

1.0  MODEL  DEVELOPMENT 

Model  development  ensures  that  the  execution  of  the  implemented  model  will 
support  the  Modeling  and  Simulation  mission.  To  fulfill  the  Model  development 
mission,  the  model  requirements  are  developed  (analyzed  and  defined);  the 
architecture  (high  level  organization/design  of  the  software)  is  developed;  the 
model  is  coded  to  specification  or  implemented;  the  model  is  tested  to  make 
certain  that  it  performs  as  intended.  Model  deficits  are  corrected  and  the  model 
is  retested;  the  model  goes  into  the  maintenance  phase  where  additional 
corrections  are  made  and/or  new  features  are  added. 

1.1  REQUIREMENTS  ANALYSIS 

Requirements  analysis  involves  four  basic  steps:  (1)  Problem  recognition  (2) 
Evaluation  and  synthesis  (3)  Specification  (4)  Review.  [4] 

1.1.1  Problem  recognition 

Problem  recognition  creates  a  careful  formulation  of  the  problem  statement. 
defines  the  issues  to  be  investigated,  understands  the  measures  of  performance 
for  evaluation  (by  the  model),  grasps  the  manner  in  which  the  model  will  be 
used,  conceives  alternative  system  configurations  of  interest  to  be  represented  by 
the  model,  and  determines  model  aggregation  (level  of  model  detail)  by  using 
experts  and  sensitivity  analysis.  The  customer/user  must  supply  input  to  foster 
this  analysis. 

1.1.2  Evaluation  and  Synthesis 

Evaluation  and  synthesis  understands  the  flow  and  structure  of  the  information 
involved,  defines  and  refines  software  functions,  establishes  system  interface 
characteristics,  and  discovers  design  constraints. 

1.1.3  Specification 

Specification  results  in  a  design  specification  (e.g.  an  SRS  or  prototype)  that 
specifies  what  is  to  be  constructed  and  validation  criteria  (the  requirements 
themselves)  used  to  test  against  to  make  certain  that  the  correct  model  is 
constructed  (if  the  criteria  are  met  then  the  model  is  valid). 
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1.1.4  Review 

Review  secures  approval  from  all  interested  parties,  e.g.  the  customer/user, 
system  engineers,  etc.,  as  to  the  correctness  and  appropriateness  (this  is  one  of 
the  ways  that  validation  and  verification  are  accomplished  )  of  the  deliverables. 

1.2  DESIGN 

Design  translates  requirements  into  a  representation  of  the  software.  The  design 
focuses  on  three  areas:  data,  software  architecture,  and  procedure. 

1.2.1  Data  Design 

Data  design  identifies  individual  data  elements,  understands  the  relationship 
among  them,  designs  efficient  data  structures  that  house  them  for  access,  and 
depicts  the  flow  of  data.  Data  design  includes  data  modeling  using  ERDs* 
identifying  data  objects,  depicting  die  data  flow  using  DFDs.  building  data 
dictionaries,  and  using  various  data  structures  (array's,  linked  lists,  DB,  etc.)  to 
store  data  elements. 

1.2.2  Software  Architectural  Design 

Software  Architectural  design  assigns  requirements  to  software  components: 
decomposes  each  component  into  process  modules  and  associated  data  structures. 
and  defines  the  communication  (interfaces')  between  the  modules. 

1.2.3  Procedural  Design 

Procedural  design  focuses  on  the  processing  details  of  each  module  individually. 
These  processing  details  include  data  required  to  begin  processing  (input  data), 
data  that  the  processing  generates  (output  data),  sequence  of  events,  exact  decision 
points  (control  flow),  repetitive  operations,  data  Qrganizatipn/structure  and  any 
special  algorithms.  The  specification  of  these  design  details  are  represented  using 
a  notation  e.g.  PDL. 


1.3  CODING 

Coding  translates  design  specifications  into  an  executable  representations.  A 
programmer  makes  this  translation  using  programming  tools  (editors,  compilers, 
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assemblers).  This  entails  producing  and  analyzing  listings*  exercising  modules, 
and  placing  the  completed  modules  under  CM. 

1.4  TESTING 

Testing  ensures  that  the  software  is  an  accurate  reflection  of  the  specification. 
The  software  is  tested  against  the  requirements  from  which  the  software  was 
developed.  Some  testing  is  done  by  programmers.  This  is  testing  of  the  module 
in  isolation  and  is  called  unit  testing.  Integration  testing  refers  to  grouping 
several  modules  that  form  a  logical  unit  and  testing  them  as  a  group.  This  testing 
is  designed  to  ensure  that  groups  interact  properly.  Integration  testing  is 
continued  (more  tested  components  are  grouped)  until  the  entire  system  is  tested. 
To  do  this  test  plans,  procedures,  and  test  cases  have  to  be  generated.  Test  results 
are  evaluated  and  defects  are  corrected.  In  many  cases  regression  testing  must  be 
performed.  The  corrected  tested  modules  must  be  placed  under  CM. 

1.5  MAINTENANCE 

Maintenance  involves  modifying  the  software  in  response  to  customer  requests  to 
fix  problems  or  to  add  new  features  and  requires  doing  all  the  things  done  in 
"Model  Development". 
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2.0  PROJECT  MANAGEMENT 

Project  management  develops  and  executes  plans,  creates  organizations,  designs 
the  managerial  and  technical  process,  and  develops  the  work  elements,  schedule, 
and  budget  to  manage  the  model  development  which  ensures  the  Modeling  and 
Simulations  mission. 

2.1  PLANNING 

Planning  develops  and  adjusts  the  software  project  management  plan. 

2.2  PROJECT  ORGANIZATION 

Project  Organization  defines  the  process  model,  creates  the  organizational 
structure,  identifies  and  defines  the  organizational  boundaries  and  interfaces, 
develops  strategies/policies,  sets  procedures  and  rules,  defines  responsibility  and 
authority,  hires,  terminates,  evaluates,  trains  personnel  and  sets  project 
responsibilities. 


2.3  MANAGERIAL  PROCESS 

Managerial  Process  identifies  management  objectives  and  priorities,  identifies 
assumptions,  dependencies,  and  constraints,  manages  risk,  sets  monitor  and 
control  mechanisms,  and  develops  a  staffing  plan. 

2.4  TECHNICAL  PROCESS 

Technical  process  identifies,  selects,  and  correctly  uses  development  processes, 
methods,  tools,  and  techniques,  writes  software  documentation,  and  facilitates 
project  support  functions. 

2.6  WORK  ELEMENTS,  SCHEDULE,  AND  BUDGET 
MANAGEMENT 

Work  elements,  schedule,  and  budget  management  identifies  and  assigns  work 
packages,  identifies  dependencies,  estimates  resource  requirements,  develops  a 
budget,  and  allocates  resources,  builds  and  adjusts  schedules. 
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3.0  VALIDATION,  VERIFICATION,  AND  CERTIFICATION 

Validation,  verification,  and  certification  supports  Models  and  Simulations 
ensuring  that  the  implemented  model  is  an  accurate  representation  and  abstraction 
of  the  situation  intended  for  study,  that  it  was  built  correctly  (implying  that  it  will 
perform  correctly),  and  that  if  used  as  intended  is  a  credible  tool. 

Validation  and  Verification  are  a  subset  of  SQA  activities  and  is  performed 
iteratively  throughout  model  development.  The  model  is  certified  after  it  has 
been  accepted  and  used  successfully  (multiple  times)  by  the  customer.  To  ensure 
a  valid  conceptual  model  the  requirements  must  be  faithfully  translated 
throughout  model  development.  Verification  happens  as  the  requirements, 
design,  software,  and  documentation  are  reviewed,  inspected,  and  tested. 
Validation  and  Verification  result  from  the  coned  application  of  methods  and 
tools,  conducting  formal  technical  reviews,  testing,  and  record  keeping  and 
reporting.  Certification  involves  showing  empirically  that  the  tool  performs  as 
intended  and  a  formal  declaration  by  an  agency  stating  the  same. 

3.1  APPLICATION  OF  METHODS  AND  TOOLS 

Application  of  methods  and  tools  monitors  the  use  of  the  methods  and  tools  used 
to  construct  the  model. 

3.2  CONDUCTING  FORMAL  TECHNICAL  REVIEWS 

Conducting  formal  technical  reviews  involves  reviewing  various  work  products 
including  requirements,  specs,  plans,  documentation,  and  designs. 


3.3  TESTING 
Testing  is  covered  above 

3.4  RECORD  KEEPING  AND  REPORTING 

Record  keeping  and  reporting  collects  the  results  of  reviews,  audits,  change 
control,  testing,  and  other  SQA  activities  and  disseminates  them  on  a  need-to- 
know  basis. 
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3.5  EMPIRICAL  DEMONSTRATIONS 

Empirical  Demonstration  involves  documentmg_the  successful  results  of  the 
model.  In  order  to  produce  valid  and  useful  Modeling  and  Simulation  results,  the 
Hata  used  in  a  model  must  be  W&C  by  the  data  producer  and  VV&C  by  the 
exercise  or  study  director  as  part  of  the  model  or  exercise  VV&A  process. 

The  results  and  conditions  of  the  demonstrations  are  used  as  the  basis  for  issuing 
the  formal  certification. 
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4.0  DATA  COLLECTION 

Data  Collection  helps  to  ensure  the  Modeling  and  Simulation  mission  by 
providing  model  developers  with  information  used  to  construct  a  model  that 
fairly  represents  the  situation,  input  data  for  the  model,  and  to  provide  insight 
into  the  application  types  where  use  of  the  model  is  indicated.  It  also  provides 
support  for  scenario  generation,  support  for  requirements  definition,  and  level 
of  model  aggregation.  To  ensure  this,  sub-mission  data  standards  and  recognized 
authoritative  data  sources  are  required  to  ensure  data  and  algorithm  consistency, 
especially  for  interoperating  across  models.  In  general  Modeling  and  Simulation 
data  collection  focuses  on  locating,  accessing,  acquiring,  and  preparing  data 
inputs.  In  support  of  this  sub-mission  users  and/or  developers  may  use  data 
centers  whose  purpose  is  to  collect,  maintain,  verify  and  validate,  and  provide 
data  either  to  specific  models  or  to  users  of  models.  In  summary,  the  Modeling 
and  Simulation  community  needs  rapid  access,  acquisition  and  processing  of 
quality,  consistent,  valid  data  for  input  to  models  and  simulations.  [5] 

4.1  MODEL  USE 

Model  use  involves  using  all  descriptive  information  about  the  model  to 
understand  if  the  model  is  appropriate  given  some  arbitrary  application  and  using 
correct,  proper,  and  appropriate  inputs  with  which  to  perturbate  the  model. 

This  information  includes,  title,  model  category,  keywords,  proponent. 

dtekper,  point  of  CQntag.ti.statemenLgf  purpgss,-de&ciip.tifln,  date 
imolemented/updated.  limitations,  input,  parameter  range  values,  expected  output, 
required,  hardware,  software.. available  documentation,  classification,  model  data 
verified  and  validated,  data  owner/authoritv.  data  standard  employed,  and  general 
data  atom  the  model, 

4.2  SCENARIO  GENERATION 

Scenario  generation  see  scenario  generation  below 

4.3  MODEL  CONSTRUCTION 

Model  construction  see  requirements  analysis  above 

4.4  MODEL  AGGREGATION 

Model  aggregation  refers  to  level  of  detail  captured  in  the  model  about  the 
situation  being  modeled.  These  activities  are  covered  above  in  requirements 
definition. 
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4.5  MODEL  INPUT 

Model  input  is  covered  above  under  model  use. 
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5.0  MODEL  DISPERSION  AND  SUPPORT 

Model  dispersion  and  support  include  training  (plans,  manuals,  procedures), 
user  documentation,  user  support ,  (help  line)  data  sharing  (including  executing 
the  simulation  and  exchanging  data  interoperably  with  other  simulations),  and 
shipping  to  facilitate  a  wide  range  of  model  use.  This  sub-mission  could  be 
supported  by  an  Information  Resource  Repository  (IRR)  which  would  logically 
contain  all  of  an  organization's  information  resources  including  tools  for 
accessing,  managing,  maintaining  and  protecting  its  resources. 

5.1  TRAINING 

Training  includes  developing  training  plans  and  procedures,  building  training 
manuals  and  other  user  documentation,  and  setting  ttaimog  jfiUcifiS- 

5.2  USER  SUPPORT 

User  support  ensures  the  mission  success  by  being  a  major  component  of  the 
support  infrastructure.  This  infrastructure  includes  the  help  line  and  the  IRR 
which  will  logically  contain  all  of  an  organization's  resources  including  tools  for 
accessing,  managing,  maintaining  and  protecting  its  resources.  The  resources 
include  entities  such  as:  data/infQrmatiOD/knQiyk.dge  Jasss;  S!2fma££  (e.g.,  models 
and  simulations,  algorithms);  data  .and  infQrmatifllLSiandaEd.S  (e.g.,  process  and 
data  models,  data  dictionaries);  data  standardization  entities  (e.g.  metadata 
describing  standard  data  entities,  attributes,  relationships,  domains/nomenclature, 
and  symbols)  directories/catalogs  of  information  resources,  organizations,  etc.; 
documents:  common  tools  to  engineer,  manage  and  manipulate  information 
resource  entities:  toolsets  that  could  include  capabilities  for:  directory  accessing, 
browsing,  linking,  etc.;  management  and  use  of  data  standards;  and  application 
tools  specialized  to  meet  the  organization's  particular  needs  management  and  use 
of  data  standards;  management  and  verification  of  models  and  simulations; 
management  and  verification  and  validation  of  databases;  management  of 
algorithms;  preprocessing  and  post  processing  of  data;  CASE  tools  for  developing 
Modeling  and  Simulation  software;  etc.  [5] 
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6.0  MODEL  OPERATION 

Model  operation  refers  to  activities  required  to  manage  an  executing  model. 
Controllers  are  required  to  make  certain  that  all  model  components  are 
synchronized,  monitors  collect  output,  coordinators  ensure  that  all  needs  are 
being  serviced,  model  operators  are  required  to  make  the  model  react.  Model 
users  may  want  to  report  their  experience  using  models  to  the  model  owners. 
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7.0  SCENARIO  GENERATION 

Scenario  generation  refer**  to  scripting  model  runs.  A  scenario  specifies  the 
operational  environment.  A  scenario  also  specifies  the  system  configurations. 
i.e.  the  values  the  various  factors  the  model  needs  to  produce  output  (input 
parameters ,  structural  assumptions),  may  assume.  Scenario  generations  are  the 
result  of  an  "Experiment  Design  Plan"  which  provide  management  support  for 
designing  model  runs  and  experiments.  It  also  provides  support  for  maintaining 
records  of  experiments. 
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8.0  OUTPUT/ANALYSIS 

Output/analysis  is  concerned  with  the  post-processing  of  Modeling  and  Simulation 
results.  The  simulations  provide  us  with  hard  answers  when  using  a  consolidated 
model.  These  models  have  been  constructed  with  data  that  has  a  high  probability 
of  being  correct  at  some  level  of  detail.  The  output  of  these  models  can  be 
considered  valid  enough  to  make  decisions  on  or  used  to  enforce/set  policy. 

When  using  an  exploratory  model  the  model  does  not  provide  hard  answers. 
Instead,  these  models  provide  additional  insight  e.g.  existence  proofs,  hypothesis 
generation,  or  special  cases.  These  models  have  been  constructed  without  data 
where  there  is  a  high  confidence  level  regarding  its  certainty. 

By  using  data  standards  for  variables  within  models  and  simulations,  the  data 
results  for  post-processing  will  be  standardized  and  more  easily  handled  by 
shareable  tools. 
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ECS  provides  toolkits  to  appropriate  SCFs 
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Guide  Software  Reuse  -  Text  search  and  hypertext  navigation  built  up 

existing  information  resource  discovery  tools 
(Wide  Area  Information  Servers  (WAIS)  and 
World- Wide-Web  (WWW)  and  X-Mosaic). 


different  commercial  packages. 
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corrupted,  disturbed,  and/or  jammed. 

-  At  the  same  time,  determine  the  realistic  Measured 

operational  boundaries  in  which  the  netted  DIS  performance 
can  still  operate  albeit  in  a  degraded  state. 
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1A.  DIS-Compliant  Run/review  quality  tests  on  data  to 

Producer  Data  W&C  ensure  ranges,  accuracies,  scalar 

Verification  values,  etc.  are  correct  and  within 

acceptable  limits 

Ensure  that  data  will  run  with  DIS 
network  without  corruption 

Examine  Producer  Data  W&C  history 
for  incorporation  into  current  User 
Data  W&C  plans 


IB.  DIS-Compliant  Review  previous  User  data  for 

Previous  User  Data  applicability 

W&C  Verification 

Examine  Previous  User  Data  W&C 
history  for  incorporation  into 
current  User  Data  W&C  plans 
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2A.  Use  Data  Standards 
to  determine  data 
model  for  intended 
use 


W  V 
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3. A  Review  &  Validate 
Appropriateness  of 
Data  for  Model  Purpose 


Review/analyze  metadata  for  each 
source  for  compatiblity  with 
proposed  model  algorithms 


Determine  availability  of  data 
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4. A  Design  Data  Algorithms  Determine  details  of  each  data 

transfer  and  message 

Develop/test  data  algorithms 
as  necessary  for  Key  algorithm 
analysis 

Determine  impact  on  timing 


/ 
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5. A  Ver.i  fy  &  Implement  Data  Verify  the  design  of  each  interface 
Algorithms  &  Integrate  including  the  DIS  PDUs  being  used 
Data 

Verify  that  messages  embedded  in 
PDUs  are  handled  correctly  IAW 
with  Data  W&C  plans 

Confirm  each  data  field,  value 
and  range 

If  necessary  develop  a  data  test 
driver  to  simulate  the  network  to 
ensure  interoperability  and 
correctness 
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6. A  Validate,  Certify  & 
Produce  Report 


Calibrate  the  contribution  of 
live  components  from  data  collected 
during  live  exercises  for  use  in 
the  test  suite  as  default  values 
for  simulating  them 

Perform  real-world  comparisons  to 
data  when  possible 

Produce  report  of  findings  for 
repository  and  for  use  in 
accreditation 


-  652  - 


TJ  * 


ffi  ffi  Cr 
>  •;  ffi 

©  ©  *“ 
O'-© 

cc  .2 


Si  m 

to  ■£  c 
■D  «g  0 

Q.  g 

o  «  .9 

ffi  ffi  ©: 
°  3  fe 

ocriS 
<  ffi  o 


ffi 

S  >  c 
w  o  o 

■°  a'-S 

ft?  I 

S  •  £ 

o  g  = 

ffi 


4-* 

(0 

<b 

kJ 

ffi 

E 

3 

rr 

o 

3 

(0 

<D 

T3 

c 

h_ 

O 

ffi 

ffi 

k. 

Q_ 

h 

<0 

o 

■o 

4-* 

TJ  «-i 

c  « 

©  *5  O* 

)=  tC  © 

(v  m  w 
8-  °  - 
^  5 


ffi 

> 

# 

>»  3S 

V) 

:t=  © 

0) 

C  'Z 

o 

w 

ffi  o 

3 

T3  £ 

O 

3 

w 

ffi 

V 


■O  ■g 
3  =  ® 

ffi  ©  3 

f  ~  2 

|  «  2 
®  o  « 


ffi 

O  C 
g-  3  ffi 
O  "O  Q. 

ffi  2 
>Q.O 
®  *7  o8 

o  S3  > 

«o  > 
■o 


/  .N 

.  ffi  ffi  ffi  V 

/  «  tS  ffi  ' 

ffi  ffi  3  . 

T3  X3  ' 

=  *D 

ffi  ■O  ffi 

>  ffi  "O 

5  ®  m 

©  ffi  £ 

~  3  C 

'  ®  ffi  ”  ' 

V  CC  w  O  / 

'  *•— 

S  ✓ 


ffi  ffi 

to  to  % 
ft  *S  ffi 

“■  ffi  J>£ 
®  TJ  o 
n  « 


2  2  « 
®  ffi  « 

ffi  o."0 

§  2  S 

o  9-  3 
ffi  Q.  O 

©  «  Cfl 

</) 


o 

ffi 

U 

3 

c 

o 

k. 

© 

T3 

o 

-4-> 

ffi 

o. 

w 

o 

© 

CL 

w 

CL 

ffi 

‘■E 

ffi 

“O 

© 

o 

»  ffi  . 

^  O  S3 

|  1  § 

S  8  O 


to 

** 

« 

■o 

C 

o 

ffi 

4 

ffi 

ffi 

</> 

o 

ffi 

3 

a 

4 3 

a> 

ffi 

CL 

o 

DRAFT  28  JUNE  1994 


-  653  - 


|  DATA  W&C  PROCESS  GUIDELINES,  NON-DIS  APPLICATIONS  1 

ACTIVITY 

PROCEDURE 

TOOLS/METHOD  | 

1.  Develop  study 
plan,  identify  data 
required. 

(Data  User) 

-  Identify: 

Threats,  Force  Structure, 
Scenario,  Time  Frame ; 
Geographic  Area,  Models, 
Study  Milestones. 

-  Determine: 

Materiel  Systems, 

Variants ,  and  Surrogates 
to  be  included  in  the 
study . 

-  Specify: 

Categories/type  of  data 
required. 

Study  I 

director  and  1 
team  working  ” 
under  the 
guidance  of 
a  Study 
Advisory 

Group  (SAG) . 

2 .  Identify 

authoritative 

sources. 

(Data  User) 

-  Determine: 
Internal/extemal 
availability  of  data. 

-  Review: 

Policies/regulations  for 
authoritative  sources. 

-  Specify: 

Authoritative  sources  for 
categories/type  of  data 
required. 

Study  team 

using 

published 

policies, 

guides,  and 

Automated 

Data 

Repositories 
(ADRs)  in 
coordination 
with  data 
producers . 

3 .  Prepare  and 
certify  data 
request. 

(Data  User) 

-  Identify: 

Specific  type,  format,  and 
range  of  data  required. 

-  Resolve: 

Data  conflicts  and 
inconsistencies . 

-  Obtain: 

Data  request  certification 
if  required. 

Study 

director  and 
other 

agencies  as 
required  by 
local  policy 
or 

regulation. 

4 .  Transmit  data 
request  to 
producer. 

(Data  User) 

-  Determine: 

Media  and  method  of  data 
request  transmission. 

Study  team 
using  ADRs, 
data 

networks  and 

other 

electronic 

media. 


-  Transmit: 

Data  request  to 
authoritative  source. 


DATA  W&C  PROCESS  GUIDELINES,  NON-DIS  APPLICATIONS  1 

ACTIVITY 

PROCEDURE 

TOOLS/METHOD 

5.  Receive  and 
review  data 
request . 

(Data  Producer) 

-  Obtain: 

Data  request  from  data 
user. 

-  Review: 

Data  request  to  verify 
correct  source, 
completeness  of 
information  in  the 
request,  and  feasibility 
of  milestones. 

Data 

producer  or 
data  manager 
using  ADRs 
and/or  other 
DBMS  to 
review 

request.  1 

6.  Accept  data 
request ,  obtain 
clarifications . 

(Data  Producer) 

-  Determine: 

Acceptability  of  data 
request. 

-  Prepare: 

Questions,  requests  for 
additional  information, 
indication  of  feasibility 
of  meeting  study 
milestones. 

Data 

producer  or 
data  manager 
using 

Subject 

Matter 

Expert  (SME) 
review. 

7.  clarify  data 
request/provide 
information. 

(Data  User) 

-  submit: 

Answers  to  questions  from 
data  producer,  additional 
information  on  study. 

Study 

director  and 
team 

responding . 

8 .  Develop  data 
producer  W&C  plan. 
(Data  Producer) 

-  Determine: 

Method  for  searching, 
extracting,  generating 
data;  final  format  of  data 
package;  applicable 
process  and  data  models; 
data  standrds  to  be 
applied  during 
verification;  best  source 
of  comparison/benchmark 
data  to  be  used  in 
validation;  resources  and 
milestones. 

Data 

producer  or 

data  manager 

following 

published 

data  1 

standards 

and 

procedures 
to  produce 
written  W&C 
plan. 
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DATA  W&C  PROCESS  GUIDELINES,  NON-DIS  APPLICATIONS  | 

ACTIVITY 

PROCEDURE 

TOOLS/METHOD 

9.  Review/ validate 
requested  data  for 
intended  use. 

(Data  Producer) 

-  Review: 

Models/applications  that 
will  use  requested  data. 

-  Determine: 

Appropriateness  of 
requested  data  for  its 
intended  use. 

Data 

producer  SME 
performing 
review  of 
intended  use 
of  requested 
data. 

-  Notify: 

Data  user  of  caveats  and 
limitations  on  requested 
data  and  possible 
Inappropriateness  of 
requested  data  for 
intended  use. 

10.  Select/generate 
appropriate  source 
data. 

(Data  Producer) 

-  Determine: 

Technique  for  extracting 
or  generating  requested 
data . 

-  Produce: 

Requested  data  in  required 
format  and  range. 

Data 

producer 
using  ADRs, 
other  DBMS, 
and  models/ 
simulations 
to  produce 
data. 

11.  Perform 
verification. 

(Data  Producer) 

-  Use: 

Techniques  and  procedures 
to  ensure  that  data  meets 
constraints  defined  by 
data  standards  and 
business  rules  derived 
from  process  and  data 
modeling. 

Data 

producer 
using 
automated 
and  manual 
methods . 

12 .  Perform 
validation. 

(Data  Producer) 

-  Determine: 

Through  subject  matter 
experts  that  the  data 
produced  represents  the 
best  available  within 
stated  criteria  and 
assumptions  and  within 
time  and  resource 
constraints. 

Data 

producer  SME 
using 
automated 
and  manual 
methods . 

13 .  Prepare  data 
package. 

(Data  Producer) 

-  Produce: 

Complete  package  of 
requested  data  and 
appropriate  meta-data  in 
the  format  and  media 
required  by  the  data  user. 

Data 

producer 
using  ADRs, 
other  DBMS 
and  manual 
methods . 

DATA  W&C  PROCESS  GUIDELINES ,  NON-DIS  APPLICATIONS 

ACTIVITY 

PROCEDURE 

TOOLS/METHOD 

14 .  Prepare  data 
producer 
certification . 

(Data  Producer) 

-  Review: 

Completed  data  package. 

-  Prepare: 

Data  producer 
certification  statement. 

Data  manager 
conducting 
senior  level 
review  and 
preparing 
written 
correspon¬ 
dence  . 

15.  Transmit  data 
and  certification 
to  user. 

(Data  Producer) 

-  Send: 

Data  package  and 
certification  statement  to 
data  user  in  requested 
format,  media,  and 
transmission  means. 

Data 

producer 
using  ADRs, 
data 

networks  and 
other 
electronic 
media. 

16.  Prepare  data 
use  certification. 
(Data  User) 

• 

-  Review: 

Data  package  and  producer 
certification. 

-  Prepare: 

Statement  indicating  that 
data  are  appropriate  for 
the  study  and  were  used 
correctly. 

Study 
director 
conducting 
senior  level 
review  and 
preparing 
written 
correspon¬ 
dence. 

(  210  )  671  -  1905  DSN  473  -  1905  FAX  (  2459  ) 
e  -  mail  :  barker  @  tecnetl.jcte.jcs.mil 
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Detailed  Signature  Data  Required  for  new 
Warning  Systems 
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Chartered  as  JTF  -Nov  92 
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SURVEY 
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-  Threat  to  a  Single  Tactical  Asset-Ship,  Plane,  Tank 

IR  (MWIR,  SWIR,  LWIR)  &  UV  Top  Priority 

-  RCS  &  RF  Well  Understood 

-  Passive  IR  &  UV  Emerging  Threat  Technologies 
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Develop  a  Plan  to  Provide  for  Formalized  DoD 
Implementation  of  TMS  Standards  and  to  Ensure 
Future  Utilization  of  the  Data  Library  by  all  the 
Services 


Program  Test  Schedule 


Planning/Logistics  Execution  Analysis 


COMMUNITY  SUPPORT 
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JTAMS  PRODUCTS 
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»  Requirements  Tracking  System 
OBJECTIVE  4===Legacy  Implementation  Plan 
»  Designated  Legacy  Manager 
»  All  Products  In  Place 
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Test  Directors  and  Engineers  Tasked  with  the  above 
Modelers  and  Intelligence  Support  to  the  above 
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-  Library  Connects  Users  with  Similar  Requirements 

-  ELIMINATE  DUPLICATE  TESTS 

Model  Improvements 

-  High  Quality  Data  to  improve  such  Models  as  SIRRM  and  SPF 
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Promotes  Efficiencies  in  Testing 


LIBRARY 

DEVELOPMENT 
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»  Some  Personnel  in  place 
-  Close  Working  Relationship  with  JTF 
»  Parallel  Hardware  Definition/Purchase 
»  JTF  Data  Base  Personnel  from  AFEWC 
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Background  —  JTAMS  data  is? 


Data  Quality  Implications 
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-  many  alternative  representations  are  possible 

»  corresponding  to  different  data  models/formats 

»  but  every  representation  is  a  model  of  an  abstract  data  view  of  the 
real  TMS 


How  does  JTAMS  Promote  TMS 

Quality? 
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How  does  JTAMS  Promote  TMS 

Quality?  (cont.) 
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-  Written  quicklook  reports  and  published  test  reports 

-  Expert  Panel  review  conferences  and  documenting  meeting 
summary  results 


Framework  for  Providing  Quality 


Data  V V  &  C  Process  for  JTAMS 

(cont.) 
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Cannot  be  complete  w/o  implementing  pilot  study 
-  Incorporates  use  or  purpose  of  data  collected 


Data  VV  &  C  Process  for  JTAMS 
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-  Propagate  rest  Its  back  to  data  maintainer  (DB  archive)  and  data 
source  (instrumentation  /  calibration  procedures) 

-  Appropriateness  depends  on  user  purpose  (study-specific) 


JTAMS  Data  Process  Improvement 
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JTAMS  Data  Process  Improvement 

Steps 
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Establishing  control  over  TMS  processes 

-  Check  conformance  to  requirements  (i.e.,  cross-calibration  ) 


JTAMS  Data  Process  Improvement 

Steps  (cont.) 
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-  Obtained  consensus  among  stakeholders  via  event  execution  and 
expert  panel  reviews 

-  Developed  and  documented  appropriate  techniques  (Handbook) 


LEGACY 
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STANDARDS  AND  PROCEDURES  PERIODICALLY  UPDATED 


THE  JTAMS  TRAIN 

(ALSO  REPRESENTS  THE  GENERAL  CASE) 


WE  CALL  THIS 
-  THE  SCIENTIFIC  METHOD 
-  CONTROL  THEORY 


Summary  of  IEEE  Metadata  Workshop 


Focused  on  Scientific  Data  &  Metadata  Needs 
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Try  to  produce  a  “reference  model” 

-  Providing  a  framework  for  scientific  metadata 

-  To  support  access  &  [semi-]  automated  machine  translation 


Metadata  should  allow  systems  to  help  users 

-  Access  data  (by  interpreting  queries) 

-  Translate  data  (by  transforming  units,  scales,  etc.) 

-  Construct  virtual,  composite  datasets 
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Begin  by  modeling 
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FOREWORD 

This  Manual  is  issued  under  the  authority  of  DoD  Directive  8320.1,  "Department  of  Defense 
Data  Administration,"  September  26,  1991.  It  provides  procedural  guidance  for  DoD  Data 
Quality  Assurance  as  established  by  DoD  8320. 1-M,  "Data  Administration  Procedures," 
September  23,  1993  (DRAFT). 

This  Manual  applies  to  the  Office  of  the  Secretary  of  Defense  (OSD);  and  to  the  Military 
Departments  (including  the  National  Guard  and  Reserve  Components),  the  Chairman  of  the  Joint 
Chiefs  of  Staff  (CJCS)  and  the  Joint  Staff,  the  Unified  and  Specified  Combatant  Commands,  the 
Inspector  General  of  the  Department  of  Defense,  the  Defense  Agencies,  and  the  DoD  Field 
Activities  (hereafter  referred  to  collectively  as  the  "DoD  Components”).  Its  provisions  are 
applicable  to  all  new  initiatives  to  develop,  modernize,  or  migrate  information  systems,  whether 
automated,  or  nor -automated. 

This  Manual  is  effective  immediately;  it  is  mandatory  for  use  by  the  OSD  and  all  the  DoD 
Components. 

Send  recommended  changes  to  the  Manual  to: 

Defense  Information  Systems  Agency 

Joint  Interoperability  and  Engineering  Organization 

Center  for  Information  Management/XD 

Data  Administration  Program  Management  Office 

701  S.  Courthouse  Road 

Arlington,  VA  22204-2199 

The  OSD  and  the  DoD  Components  may  obtain  copies  of  this  Manual  through  their  own 
publications  channels.  Defense  contractors  and  other  Federal  Agencies  may  obtain  copies  from: 

Defense  Technical  Information  Center  (DTIC) 

Building  S,  Cameron  Station 
Alexandria,  VA  22034-6125 

Commercial  Telephone:  1-800-225-DTIC  (1-800-225-3842) 
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The  public  may  obtain  copies  from: 

U.S.  Department  of  Commerce 

National  Technical  Information  Service  (NTIS) 

528 5  Por  Royal  Road 

Springfield,  VA  22161 

Commercial  Telephone:  1-703-487*4650 
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1987 


(x)  DoD  Directive  5200.28,  "Security  Requirements  for  Automated  Information  Systems," 
March  21,  1988 
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L  Accessibility.  The  ease  to  approach,  enter,  or  to  obtain,  (reference  (g)) 

2.  Accuracy.  Correct  data  that  conforms  to  models  derived  to  support  enterprise  requirements 
and  standards,  (reference  (d)) 

3.  Adherence  to  standards.  TO  BE  DETERMINED, 

4.  Amount  of  historic  data.  TO  BE  DETERMINED. 

5.  "As  is"  acnvitv/data  model.  Activity/data  model  that  portrays  how  a  business  process  in 
cunently  structured.  It  is  used  to  establish  a  baseline  for  subsequent  functional  process 
improvement  activities  or  programs.  See  modeling,  (reference  (d» 

6.  Authorization-  The  rights  granted  to  a  user  to  access,  read,  modify,  insert,  or  delete  certain 
data,  or  to  execute  certain  programs,  (reference  (e)) 

7.  Automated  Information  System  (AIS1.  A  combination  of  computer  hardware  and  computer 
software,  data  and/or  telecommunications,  that  performs  functions  such  as  collecting,  processing, 
storing,  transmitting,  and  displaying  information.  Excluded  are  computer  resources,  both 
hardware  and  software,  that  are:  physically  pan  of,  dedicated  to,  or  essential  in  real  time  to  the 
mission  performance  of  weapon  systems;  used  for  weapon  system  «peei»ii™H  training, 
simulation,  diagnostic  test  and  maintenance,  or  calibration;  or  used  for  research  and  development 
of  weapon  systems,  (reference  (d))  (Modified  from  DoDD  8120.1) 

8.  Availability-  The  state  when  data  are  in  the  place  needed  by  the  user,  at  the  time  the  user 
needs  them,  and  in  the  form  necessary  by  the  user,  (reference  (u)) 

9-  Bcllgvability.  To  have  faith  or  confidence.  The  extent  to  which  accept  as  true  or  real, 
(reference  (g)) 

10.  Completeness-  The  quality  of  maintaining  data  which  satisfies  all  demands  or  requirements, 
(reference  (d)) 

11-  Confidentiality-  The  concept  of  holding  sensitive  data  in  confidence,  limited  to  an 
appropriate  set  of  individuals  or  organizations,  (reference  (u)) 

12.  Consistency-  Data  is  maintained  so  that  it  is  free  from  variation  or  contradiction, 
(reference  (d)) 
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13.  Correctness.  Free  from  error  or  fault;  true  or  accurate.  Conforming  to  standards;  proper, 
(reference  (g)) 

14.  Currency.  Currency  is  a  measure  of  the  degree  to  which  specified  values  are  up-to-date, 
(reference  (h)) 

15.  Qua.  A  representation  of  facts,  concepts,  or  instructions  in  a  formalized  manner  suitable 
for  communication,  interpretation,  or  processing  by  humans  or  by  automatic  means,  (reference 

(e)) 

16.  Data  Administration  fDAdml.  The  responsibility  for  definition,  organization,  supervision, 
and  protection  of  data  within  an  enterprise  or  organization,  (reference  (a)) 

17.  Data  Administrator  (DAdl.  A  person  or  group  that  ensures  the  utility  of  data  used  within 
an  organization  by  defining  data  policies  and  standards,  planning  for  the  efficient  use  of  data, 
coordinating  data  structures  among  organizational  components,  performing  logical  database 
designs,  and  defming  data  security  procedures,  (reference  (v)) 

18.  Data  Element.  A  named  identifier  of  each  of  the  entities  and  their  attributes  that  are 
represented  in  a  database,  (reference  (e)) 

19-  Data  Element  Standardization.  The  process  of  documenting,  reviewing,  and  approving 
unique  names,  definitions,  characteristics  and  representations  of  data  according  to  established 
procedures  and  conventions,  (reference  (k)) 

20.  Data._Elcmcnt  Standardization  Life  Cycle.  Generic  elements  and  standard  data  elements 
evolve  through  one  or  more  phases  of  a  data  element  life  cycle.  The  five  phases  of  the  generic 
dement  or  standard  data  element  life  cycle  are  developmental,  candidate,  approved,  modified, 
ar<d  archived,  (reference  (k)) 

2 1  •  Data  Entity.  An  object  of  interest  to  the  organization,  usually  tracked  by  an  automated 
system,  (reference  (w)) 

22.  Data  Integrity.  The  state  that  exists  when  data  is  handled  as  intended  and  is  not  exposed  to 
accidental  or  malicious  modification,  destruction,  or  disclosure,  (reference  (e))  In  information 
processing,  the  condition  in  which  data  is  accurate,  current,  consistent,  and  complete,  (reference 
(v)) 

23.  Data  Model.  In  a  database,  the  user's  logical  view  of  the  data  in  contrast  to  the  physically 
stored  data,  or  storage  structure.  A  description  of  the  organization  of  data  in  a  manner  that 
reflects  the  information  structure  of  an  enterprise,  (reference  (e)) 
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24.  nata  Quality.  The  correctness,  timeliness,  accuracy,  completeness,  relevance,  and 
accessibility  that  make  data  appropriate  for  use.  (reference  (e)) 

25.  nafa  fti»TYttitnrv.  A  specialized  database  containing  information  about  data,  such  as 
meaning,  relationships  to  other  data,  origin,  usage,  and  format,  including  the  information 
resources  needed  by  an  organization,  (reference  (d)) 

25.  Data  Steward.  The  person  or  group  that  manages  the  development,  approval,  and  use  of 
data  within  a  specified  Functional  Area,  ensuring  that  it  can  be  used  to  satisfy  dau  requirements 
throughout  the  organization,  (reference  (k)) 

27.  Data  Synchronization.  The  timing  requirements  of  a  data  element,  or  between/among  data 
elements,  (reference  (d» 

28.  Database.  A  collection  of  interrelated  data,  often  with  controlled  redundancy,  organized 
according  to  a  schema  to  serve  one  or  more  applications;  the  data  are  stored  so  that  they  can  be 
used  by  different  programs  without  concern  for  the  data  structure  or  organization.  A  common 
approach  is  used  to  add  new  data  and  to  modify  and  retrieve  existing  data,  (reference  (e)) 

29.  Database  Administration  (DBAdm).  The  activity  responsible  for  the  enforcement  of  the 
policies  and  standards  established  by  the  DAd,  to  include  providing  technical  support  for 
physical  database  definition,  design,  implementation,  maintenance,  integrity,  and  security;  and 
coordinating  with  computer  operations  technicians,  system  developers,  vendors,  and  users. 
DBAdm  is  oriented  toward  technical  support  for  databases  and  the  effective  and  efficient  use  of 
information  technology  resources,  (reference  (d)) 

30.  Database  Administrator  (DBAdl.  A  person  or  group  which  provides  technical  support  for 
one  or  more  databases,  by  defining  database  schemas  and  subschemas,  by  maintaining  data 
integrity  and  concurrency,  providing  physical  database  design  for  performance  optimization,  and 
enforcing  the  policies,  standards,  and  procedures  set  by  the  DAd.  (reference  (v)) 

31.  Extensibility.  The  capability  to  create  new  functionality  in  data/information  system, 
(reference  (e)) 

32.  Flexibility.  Adaptability.  Responsiveness  to  change,  (reference  (g))  (Also  see 
"modularity"). 

33.  Functional  Process.  A  well-defined  (or  definable)  set  of  logically  related  tasks  and  decisions 
within  a  functional  activity  that  use  resources  to  produce  products  or  services,  (reference  (d)) 


XI 


February  4,  1994 


-  710  - 


DoD  8320.1-M-3 


34.  Functional  Prnrj»*<  Improvement  fFPIl.  Application  of  a  structured  methodology  to  define 
a  function's  "as  is"  and  "to  be"  environments;  current  and  future  mission  needs  and  end  user 
requirements;  objectives  and  a  strategy  for  achieving  those  objectives;  and  a  program  of 
incremental  and  evolutionary  improvements  to  processes,  data,  and  supporting  AISs  that  are 
implemented  through  functional,  technical,  and  economic  analysis  and  decision-making, 
(reference  (q)) 

35.  information.  Any  communication  or  reception  of  knowledge  such  as  facts,  data,  or 
opinions,  including  numerical,  graphic,  or  narrative  forms,  whether  oral  or  maintained  in  any 
medium,  including  computerized  databases,  paper,  microform,  or  magnetic  tape,  (reference  (b)) 

36.  Information  Systems  (IS).  The  organized  collection,  processing,  maintenance,  transmission, 
and  dissemination  of  information,  in  accordance  with  defined  procedures,  whether  automated  or 
manual,  (reference  (x))  (As  modified  by  OMB  Cir  A-130) 

37.  Internretabilitv.  The  ability  to  represent  or  render  the  meaning  of.  The  ability  to  extract 
an  explanation,  (reference  (g)) 

38.  Logical  Data  Model.  A  model  of  data  which  represents  the  inherent  structure  of  that  data 
and  is  independent  of  individual  applications  of  the  data  and  also  of  the  software  or  hardware 
mechanisms  which  are  employed  in  representing  and  using  the  data,  (reference  (d)) 

39.  Metadata.  Information  describing  the  characteristics  of  data;  data  or  information  about  data; 
descriptive  information  about  an  organization's  data,  data  activities,  systems,  and  holdings, 
(reference  (v)) 

40.  Metric.  A  process  of  algorithm  that  may  involve  statistical  sampling,  mathematical 
computations,  and  rule-based  inferencing.  Metrics  provide  the  capability  to  detect  and  report 
defects  within  a  sample  (reference  (n)) 

41.  Misratipn-Satcm-  An  existing  AIS,  or  a  planned  and  approved  AIS,  that  has  been  officially 
designated  to  support  standard  processes  for  a  functional  activity  applicable  DoD-wide  or 
Component- wide,  (reference  (d))  (DoDD  8120.1) 

42.  Modeling.  Application  of  a  standard,  rigorous,  structured  methodology  to  create  and 
validate  a  physical,  mathematical,  or  otherwise  logical  representation  of  a  system,  entity, 
phenomenon,  or  process.  Process  improvement  modeling  defines  and  documents  the  current  ("as 
is")  and  desired  future  ("to  be")  processes  and  information  requirements  of  a  functional  activity, 
(reference  (d)) 
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43.  Modularity.  The  extent  to  which  a  system  is  composed  of  modules,  (reference  (e)). 
Designed  with  standardized  units  or  dimensions  for  flexible  use.  (reference  (£))  (Also  see 
"flexibility"). 

44.  Physical  Data  Model.  A  representation  of  the  technologically  independent  information 
requirements  in  a  physical  environment  of  hardware,  software,  and  network  configurations 
representing  them  in  die  constraints  of  an  existing  physical  environment,  (reference  (e)) 

45.  Precision.  A  measure  of  the  ability  to  distinguish  between  nearly  equal  values;  for  example, 
four-place  numerals  are  less  precise  than  six-place  numerals;  nevertheless,  a  properly  computed 
four-place  numeral  may  be  more  accurate  than  an  improperly  computed  six-place  numeral, 
(reference  (e)) 

46.  Program  Administration.  The  management  activity  necessary  to  manage  a  program  across 
functional  and  organizational  areas,  (reference  (d» 

47.  Quality  Assurance  (QAt.  The  policies,  procedures  and  systematic  actions  established  in  an 
enterprise  for  the  purpose  of  providing  and  maintaining  some  degree  of  confidence  in  data 
integrity  and  accuracy  throughout  the  life  cycle  of  the  data.  The  planned  systematic  activities 
necessary  to  ensure  that  a  component,  module,  or  system  conforms  to  established  technical 
requirements,  (reference  (e)) 

48.  Relatabilitv.  The  quality  of  data  that  permits  it  to  be  rationally  correlated  or  compared  with 
other  similar  or  like  data,  (reference  (d)) 

49  •  Rclsvancc/Rfilsvansy.  The  state  of  maintaining  data  in  a  condition  that  provides  the  ability 
to  retrieve  the  specific  information  needed  by  the  user,  (reference  (d)) 

50.  Reliability-  The  extent  to  which  can  be  relied  upon;  dependability,  (reference  (g)) 

51.  Schema.  A  description,  or  global  model,  of  the  structure  of  a  database,  (reference  (e)) 


element,  (reference  (d)) 


The  organization(s)  responsible  for  entering  data  values  for  a  data 


53.  Stability.  Constancy  of  character  or  purpose;  steadfastness.  Reliability;  dependability, 
(reference  (g)) 

54.  Technical  Infrastructure.  The  internal  framework  that  must  be  built  to  implement  an 

.. _ _ t _ *  _  r  r  *  v  * 


operational  service,  (reference  (d)) 
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55.  Timeliness,  a  condition  which  requires  that  a  data  item  or  multiple  items  are  provided  at 
the  time  required  or  specified,  (reference  (d)) 

56.  "To  Be"  Activitv/Data  Model.  Activity/data  models  that  result  from  a  functional  process 
improvement  action  or  program.  The  “to-be"  model  shows  how  the  business  process  will 
function  and  the  data  it  will  use  after  the  improvement  action  is  implemented.  See  modeling, 
(reference  (d)) 

57.  Traceability.  To  ascertain  the  successive  stages  in  the  development  or  progress  of.  To  have 
origins;  be  traceable,  (reference  (g)) 

58.  Unit  of  Measure.  TO  BE  DETERMINED. 

59.  Uniqueness.  The  state  of  being  the  only  one  of  its  land;  sole.  Being  without  an  equal  or 
equivalent;  unparalleled,  (reference  (g» 

60.  Validity.  The  quality  of  maintained  data  that  is  found  on  an  adequate  system  of 
classification  (e.g.,  data  model)  which  is  rigorous  enough  to  compel  acceptance,  (reference  (d)) 
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ABBREVIATIONS  AND/OR  ACRONYMS  1 
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AIS 

Automated  Information  System  D 

AIS  PM 

Automated  Information  System  Program  Manager  1 

. 

ASD  (C3D 

Assistant  Secretary  of  Defense  for  Command,  Control,  Communications.  1 

and  Intelligence  ■ 

■ 

■ 

CD  Ad 

Component  Data  Administrator  || 

CZM 

Center  for  Information  Management  1 

acs 

Chairman,  Joint  Chiefs  of  Staff  1 

DAd 

Data  Administrator  1 

DAdm 

Data  Administration  I 

DAPM 

Data  Administration  Program  Manager  E 

r,V  . 

DAPMO 

Data  Administration  Program  Management  Office  ■ 

"  •  ‘ 

DASD  (IM) 

Deputy  Assistant  Secretary  of  Defense  for  Information  Management  a 

v- 

DASP 

Data  Administration  Strategic  Plan  1 

DBAd 

Database  Administrator  ■ 

DBAdm 

Database  Administration  1 

.  ’.•**' 

DBMS 

Database  Management  System  H 

DDI 

Director  for  Defense  Information  1 

.- r;  .1' ' 

DDI  CFIM) 

Director  for  Defense  Information,  Functional  Information  Manager  fl 

•.  <■ 

1 

DDRS 

Defense  Data  Repository  System  1 
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DISA 

Defense  Information  Systems  Agency 

DoD 

Department  of  Defense 

Do D  DAd 

DoD  Data  Administrator 

DQB 

Data  Quality  Engineering 

DTR 

Data  Trouble  Report 

FAPM 

Functional  Activity  Program  Manager 

EDAd 

Functional  Data  Administrator 

FIM 

Functional  Information  Manager 

FIPS 

Federal  Information  Processing  Standards 

FPI 

Functional  Process  Improvement 

3UEF 

Integrated  Computer-Aided  Manufacturing  Design 

IM 

Information  Management 

IS 

Information  System 

OSD 

Office  of  the  Secretary  of  Defense 

OSD  PSA 

Office  of  the  Secretary  of  Defense,  Principal  Staff  Assistant 

NBS 

National  Bureau  of  Standards 

NGT 

Nominal  Group  Technique 

PM 

Program  Manager 

PQC 

Poor-Quality  Cost 

QA 

Quality  Assurance 
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Quality  Function  Deployment 
Total  Dan  Quality  Management 
Total  Quality  Management 
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CHAPTER  1 

GENERAL  INFORMATION 


A.  INTRODUCTION 


1.  This  Manual  provides  procedures  for  implementation  of  the  policies  and  concepts  set 
forth  in  DoD  Directive  8320.1,  "DoD  Data  Administration"  (reference  (a)).  This  Manual 
supports  the  DoD  Directive  8000.1,  "Defense  Information  Management  (IM)  Program” 
(reference  (b)).  It  is  one  in  a  series  of  manuals  which  describe  DoD  data  administration  (DoD 
DAdm)  procedures.  This  Manual  specifically  addresses  DoD  Data  Quality  Assurance 
Procedures  Data  are  principal  DoD  resources,  and  having  accurate,  quality  data  is  critical  to 
the  military.  Within  DoD,  satisfying  the  data  requirements  Of  the  warfighters  and  the  business 
areas  is  essential.  This  Manual  provides  the  framework  for  improving  and  ensuring  the  quality 
of  DoD  data. 


2.  The  mission  of  DoD  DAdm  is  ‘to  provide  for  effective  and  economic  acquisition, 
dissemination,  and  use  of  data  to  enhance  performance  and  integration  of  operations”  as 
documented  in  the  DoD  Data  Administration  Strategic  Plan  (DASP)  (reference  (c». 


a.  The  DoD  Data  Administration  Program  mission  concentrates  on  six  (6)  major 
DAdm  goals.  Each  goal  is  a  broad  statement  of  long-term  priority  objectives  for  DoD  DAdm. 
These  six  (6)  goals  are  to:  (1)  Support  the  Operational  Central  Repository,  (2)  Establish  Standard 
Data.  (3)  Expand  Use  of  Common  Pr^'  s  and  Tools,  (4)  Establish  Data  Quality  and  Data 
Security  Process,  (5)  Expand  Edic  c  >  mg,  and  Consultation  Services,  and  (6)  Develop 
an  Effective  Data  Administratis,  li.x  These  goals  focus  on  benefits  necessary  to 

realize  the  future  vision  of  Do”1  UAd  "  >  it-,  mid-,  and  loi.g-term  objectives  for  the  DoD 
DAdm  Program  are  documents  „nu:  \  the  DoD  Data  Administration  Strategic  Plan 

(DASP)  (reference  (c)). 


b.  Goal  No.  4:  Establish  Data  Quality  and  Data  Security  Process.  A  data  quality 
assurance  and  data  security  program  ensures  that  DoD  operations  and  decision  making  are 
supported  with  data  meeting  needs  of  availability,  accuracy,  timeliness,  integrity,  and  need-to- 
know  requirements  (reference  (c)).  One  of  the  guiding  principles  of  DoD  DAdm  is  to  ensure 
that  data  products  will  be  managed  throughout  the  life  cycles  to  improve  business  methods, 
efficiency  of  operations,  and  the  quality  of  data.  DoD  personnel  will  use  quality  data  for 
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planning  and  analysis;  as  a  result,  decision  making  will  be  improved.  This  Manual  supports  the 
DoD  DAdm  goal  to  establish  DoD  Data  Quality  Assurance  Procedures. 


B.  PURPOSE 


Tc  provide  the  highest  quality  data  to  the  DoD  data/information  community,  this  Manual 
identifies  quality  assurance  issues  and  provides  guidance  for  DoD  implementation.  Also,  this 
Manual  provides  detailed  procedures  for  the  specific  DAdm  activity  of  data  quality  assurance. 
Figure  1*1  illustrates  the  relationship  of  this  Manual  (8320.1*M*3)  to  the  DoD  Data 
Administration  policy  directive,  (DoDD  8320.1,  reference  (a)),  and  the  overall  DoD  Data 
Administration  Procedures  (DoD  8320. 1-M,  reference  (d)). 


C.  lCQPE.AND...AEPU,CABILrn: 


The  scope  and  applicability  of  this  Manual  are  identical  with  the  Scope  and  Applicability 
statements  of  DoDD  8320.1  (reference  (a)). 

D.  OBJECTIVES 


It  is  the  intent  of  DoD  DAdm  that  the  implementation  of  these  procedures  will  lead  to  useful, 
suitable,  available  and  accessible  information  to  enable  the  successful  execution  of  the  missions 
of  the  Department. 


1-2 


February  4,  1994 


POLICY  ANNUAL  PLAN 


-  718  - 


-3 


February  4,  1994 


Figure  1-1  C or  tofDoDS320.1-M-3 


-  719  - 


DoD  8320.1-M-3 


£.  ORGANIZATION  OF  THE  MANUAL 


1.  The  DoD  Data  Quality  Assurance  Procedures  encompasses  the  process,  methodology, 
tools  and  techniques  which  have  evolved  from  the  Total  Quality  Management  (TQM)  and  Data 
Quality  disciplines. 


2.  As  Figure  1*2  describes,  this  Manual  is  organized  into  four  major  pans.  Chapters  1 
and  2  provide  an  introduction  to  the  purpose  of  this  Manual  and  the  general  concepts  of  data 
quality  assurance.  Chapter  3  introduces  the  DoD  Total  Data  Quality  Management  (TDQM) 
process.  Chapter  4  provides  the  methodology  for  implementing  DoD  Data  Quality  Engineering 
(DQE)  methodology,  which  Chapter  3  introduces.  Chapter  5  provides  specific  tools  and 
techniques  for  performing  the  TDQM  process  and  the  DQE  methodology. 
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CHAPTER? 

DATA  QUALITY  ASSURANCE  CONCEPTS 


A.  INTRODUCTION 


1.  A  data  quality  assurance  program  is  integral  to  DAdm  and  ensures  that  DoD 
operations  and  decision  maldng  are  supported  with  data  that  meets  needs  in  terms  of  availability, 
accuracy,  timeliness,  and  integrity  (DoD  8320. 1-M,  reference  (d)).  Decision  support  systems 
will  use  quality  data  for  planning  and  analysis.  As  a  result,  decision-making  will  be  improved 
and  information  systems  will  be  easier  to  use.  Transactions  and  the  exchange  of  technical  and 
management  information  will  be  handled  more  quickly  and  accurately.  In  turn,  a  cost-effective 
operation  and  low  overhead  will  be  maintained  (reference  (d)>. 


B.  DAIAINTECRTIX 


1.  Data  integrity  is  defined  in  FIPS  PUB  11-3,  "American  National  Dictionary  for 
Information  Systems,"  (reference  (e))  as: 

"The  state  that  exists  when  data  is  handled  as  intended  and  is  not 
exposed  to  accidental  or  malicious  modification,  destruction,  or 
disclosure. " 


2.  This  Manual  addresses  the  validation  and  improvement  of  the  quality  of  DoD  data. 
Appropriate  application  of  data  security  provides,  at  a  minimum,  the  integrity,  availability, 
confidentiality,  and  quality  of  an  organization’s  resources.  Data  administrators  (DAds),  in 
assuring  integrity,  availability,  and  confidentiality,  shall  adhere  to  the  DoD  security  guidance 
documents. 
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c.  standard  DEFINITION  OF  DATA  QUALITY 

1.  FIPS  PUB  11*3  (reference  (e))  defines  data  quality  as: 

"The  correctness,  timeliness,  accuracy,  completeness,  relevance, 
and  accessibility  that  make  data  appropriate  for  use." 


a.  Data  quality  also  incorporates  the  use  of  activity  models,  data  models, 
entities,  attributes,  metadata,  metametadata,  schemas,  and  data  architectures. 


2.  Data  have  several  special  properties  which  distinguish  them  from  other  products. 
Unlike  manufactured  products,  data  are  intangible.  Data  have  representations,  but  no  actual 
physical  manifestations.  Since  data  are  intangible,  they  lack  directly  measurable  properties. 
Data  meaning  depends  on  the  context  in  which  they  are  used.  For  example,  a  data  item  called 
"provider"  may  mean  the  name  of  the  insurance  provider  when  used  in  a  benefits  tracking 
system  or  may  mean  the  name  of  the  health  care  professional  who  provides  services  to  a  patient 
when  used  in  a  hospital  administration  system.  The  lifetimes  of  data  are  indeterminate.  Data 
do  not  deteriorate  as  they  age  and  their  pcwr.tial  lifetimes  are  very  long.  Finally,  data  are  very 
susceptible  to  obsolescence.  Data  values  change  over  time  and  any  lags  between  cognizance  of 
the  change  and  the  actual  update  of  the  data  value  means  that  there  are  periods  when  the  data 
value  is  incorrect.  Because  data  are  very  different  from  other  products,  they  have  their  own 
quality  characteristics  ("Data  Quality  Engineering  Study",  reference  (f)).  In  the  following 
sections,  characteristics  of  data  quality  are  presented. 


D.  QUALITY  ASSURANCE 


1.  Quality  assurance  is  defined  in  FIPS  PUB  11*3  (reference  (e»  as: 

"The  policies,  procedures  and  systematic  actions  established  in  an 
enterprise  for  the  purpose  of  providing  and  maintaining  some 
degree  of  confidence  in  data  integrity  and  accuracy  throughout  the 
life  cycle  of  the  data" . 
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2.  In  order  to  assure  quality  data:  (1)  predetermined  requirements  for  excellence  must 
be  established,  and  (2)  the  data  must  conform  to  these  requirements.  Table  2-1  is  a  broad  list 
of  the  requirements  of  quality. 


ACCESSIBILITY 

ACCURACY 

ADHERENCE  TO  STANDARDS 

AMOUNT  OF  HISTORIC  DATA 

AUTHORIZATION 

AVAILABILITY 

BELEEV  ABILITY 

COMPLETENESS 

CONFIDENTIALITY 

CONSISTENCY 

CORRECTNESS 

CURRENCY 

EXTENSIBILITY 


FLEXEBIUTY/MODULARITY 

INTEGRITY 

INTERPRET  ABILITY 

PRECISION 

REUSABILITY 

RELEVANCE/RELEVANCY 

RELIABILITY 

STABILITY 

TIMELINESS 

TRACEABILITY 

UNIT  OF  MEASURE 

UNIQUENESS 

VALIDITY 


Table  2-1  Requirements  of  Quality 


a.  Not  all  data  must  meet  all  of  these  requirements.  Business  requirements  impose 
different  quality  requirements  on  data.  Table  2-2  is  the  DoD  core  set  of  requirements  for  data 
quality. 
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Category  of  Requirement 

Descriptions 

Accuracy 

(1)  A  quality  of  that  which  is  free  of  error  (ISO).  (2)  A 
qualitative  assessment  of  freedom  from  error,  with  a 
high  assessment  corresponding  to  a  small  error  (ISO). 
(FIPS  PUB,  reference  (e)) 

(2)  Correct  data  that  conforms  to  models  derived  to 
support  enterprise  requirements  and  standards.  (DoD 

8320. 1*M,  reference  (d)). 

(3)  The  quality  or  state  of  being  accurate;  deviating  only 
slightly  or  within  acceptable  limits  from  a  standard. 
(American  Heritage,  reference  (g)) 

(4)  Accuracy  is  "a  measure  of  the  degree  of  agreement 
between  a  data  value  (or  set  of  values)  and  a  source 
assumed  to  be  correct*.  (-Data  Quality  Management 
and  Technology",  reference  (h)) 

Completeness 

(1)  The  quality  of  maintaining  data  which  satisfies  all 
demands  or  requirements,  (reference  (d)) 

(2)  Having  all  characteristics;  whole,  (reference  (g)) 

(3)  Completeness  is  "the  degree  to  which  values  are 
present  in  the  attributes  that  require  them".  ("Data 

Quality  Foundation",  reference  (i)) 

Consistency 

(1)  Data  is  maintained  so  that  it  is  free  from  variation 
or  contradiction,  (reference  (d)) 

(2)  Agreement  or  logical  coherence  among  things, 
(reference  (g)) 

(3)  Consistency  is  "a  measure  of  the  degree  to  which  a 
set  of  data  satisfies  a  set  of  constraints",  (reference  (h)) 

Table  2-2  DoD  Core  Set  of  Quality  Requirements 
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Category  of  Requirement 

Descriptions 

Reliability 

(1)  The  quality  of  data  that  permits  it  to  be  rationally 
correlated  or  compared  with  other  similar  or  like  data. 
(DoD  8320. 1>M,  reference  (d)) 

(2)  To  bring  into  logical  or  natural  association.  To 
interact  with  others  in  a  meaningful  or  coherent  fashion. 
(American  Heritage,  reference  (g)) 

Relevancy 

(1)  The  state  of  maintaining  data  in  a  condition  that 
provides  the  ability  to  retrieve  the  specific  information 
needed  by  the  user,  (reference  (d)) 

(2)  Pertinent  to  the  matter  at  hand.  The  capability  of  an 
information  retrieval  system  to  select  and  retrieve  data 
appropriate  to  a  user's  needs,  (reference  (g» 

Timeliness 

(1)  A  condition  which  requires  that  a  data  item  or 

multiple  items  are  provided  at  the  time  required  or  \ 

specified,  (reference  (d))  I 

(2)  Occurring  at  a  suitable  or  opportune  time;  well-  1 

timed,  (reference  (g)) 

(3)  Currency  is  "a  measure  of  the  degree  to  which 
specified  data  values  are  up  to  date"  ("Data  Quality 
Management  and  Technology",  reference  (h)).  In  some 
cases,  timeliness  is  a  synonym  for  currency  ("Data 

Quality  Foundation",  reference  (i)).  In  others, 
timeliness  has  a  completely  different  meaning  -  see 
timely  accessibility.  Timely  accessibility  refers  to  "the 
response  to  a  user  request  for  data  should  be  returned  in 
a  timely  fashion.  If  the  system  does  not  return  the  data 
when  the  user  needs  it,  it  mav  not  be  used"  ("Managing  1 
The  Data-Base  Environment' ,  reference  0))-  | 

Table  2-2  DoD  Core  Set  of  Quality  Requirements  (continued) 
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Category  of  Requirement 


Descriptions 


Uniqueness 


Validity 


(1)  The  state  of  being  the  only  one  of  its  kind;  sole. 
Being  without  an  equal  or  equivalent;  unparalleled. 
(American  Heritage,  reference  (g)) _ 

(1)  The  quality  of  maintained  data  that  is  found  on  an 
adequate  system  of  classification  (e.g.,  data  model) 
which  is  rigorous  enough  to  compel  acceptance.  (DoD 
8320. 1-M,  reference  (d)) 


(2)  The  state  or  quality  of  being  sound.  Producing  the 
desired  results.  (American  Heritage,  reference  (g)) 


Table  2-2  DoD  Core  Set  of  Quality  Requirements  (continued) 


E.  THE  DATA  LIFE-CYCLE 


1.  DoD  must  manage  the  quality  of  data  throughout  the  data  life-cvcle.  Appendix  A, 
"Life-Cycle  Management  of  Data",  of  DoD  8320. 1-M  (reference  (d))  identifies  the  stages  of  the 
data  life-cycle.  These  stages  are: 

(a)  Identification 

(b)  Standardization 

(c)  Acquisition 

(d)  Maintenance 

(e)  Archival 
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2.  The  identification  and  standardization  phases  identify  and  document  data  quality 
requirements  for  all  data  elements.  Data  quality  requirements  are  defined  from  various 
authoritative  sources  during  the  identification  and  standardization  phases  of  the  data  life-cycle. 
DAdm  products,  such  as  data  models,  the  Defense  Data  Repository  System  (DDRS),  and  reverse 
engineering  documentation  document  these  requirements. 


a.  The  final  product  of  the  standardization  phase  is  the  approved  DoD  standard 
data  and  metadata  including  entities,  attributes,  definitions,  and  data  values.  These  data  element 
standardization  procedures  provide  the  necessary  framework  to  facility  data  exchange, 
maximize  data  sharing  opportunities  throughout  the  DoD,  and  enforce  dam  standards. 


b.  Metadata  can  be  viewed  as  a  type  of  controlling  fiamework  that  outlines  a 
broad  set  of  rules  to  which  the  data  of  interest  to  an  enterprise  must  be  in  compliance.  Data 
quality  assurance  is  applied  to  the  development  of  metadata.  DoD  8320.1-M-l,  "DoD  Data 
Element  Standardization  Procedures."  (reference  (k))  and  DoD  832Q.1-M-X,  "DoD  Data  Model 
Development,  Approval,  and  Maintenance,"  (reference  0))  dictate  the  specific  requirements  and 
rules  to  ensure  the  quality  of  data  models  and  standard  data  elements. 


3.  During  the  acquisition  and  maintenance  phases  of  the  data  life-cycle,  DoD 
administrators  (DBAd)  develop  and  maintain  physical  data  models  based  on  approved  logical 
models.  They  develop  and  maintain  database  structure  using  approved  entities  and  attributes. 
DoD  8320.1-M'4,  DoD  Database  Administration,"  (reference  (m))  documents  these  database 
administration  (DBAdm)  procedures.  Data  quality  assurance  is  applied  to  the  design, 
implementation  and  maintenance  of  physical  databases. 


a.  The  acquisition  and  maintenance  phases  ensure  that  data  quality  requirements 
are  implemented  properly  in  databases  and  application  software.  Technical  development 
activities  must  develop  rule  sets  and  quality  metrics  (i.e.,  standards  of  measures)  from  data 
quality  requirements  to  design  filters  (i.e.,  edit  checks)  in  new  databases,  application  software, 
and  information  systems  (IS).  Procedures  for  developing  these  rule-sets  and  quality  metrics  are 
discussed  in  Chapter  4  of  this  Manual.  Users  must  perform  data  quality  analysis  to  detect  and 
prevent  data  value  defects  before  they  corrupt  databases  or  end  user  applications.  Where  filters 
do  not  exist  in  legacy  systems,  the  data  must  be  extracted  and  examined  manually  or  downloaded 
and  analyzed  with  a  data  quality  tool  designed  to  generate  the  necessary  filters.  Any  defects 
found  as  a  result  of  the  data  analysis  must  be  noted  and  forwarded  for  correction. 
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b.  During  the  acquisition  and  maintenance  stages,  these  Data  Quality  Assurance 
Procedures  are  applied  to  the  data  values  in  both  DoD  automated  systems  arid  data  repositories. 
Therefore,  a  data  quality  baseline  shall  be  established  and  measured  for  all  DoD  automated 
informadon  systems  (AIS)  and  data  repositories  using  quality  metrics.  These  data  quality 
baselines  are  used  to  measure  performance  against  target  goals  for  data  quality. 


c.  When  data  are  no  longer  current,  they  are  generally  archived  depending  upon 
the  requirements  of  the  mission.  Archived  data,  while  not  current,  are  still  useful,  and  are 
somedmes  required  by  law  or  regulations.  It  is  especially  important  that  the  data  be  of  the 
highest  quality  prior  to  being  archived.  By  nature,  there  is  less  institutional  knowledge 
regarding  archived  data;  therefore,  to  be  useful  the  archived  data  must  be  of  high  quality. 


F.  QUALITY  METRICS 


1 .  Data  quality  metrics  provide  the  fundamental  measurement  capabilities  that  form  the 
Joundation  of  a  data  quality  assurance  process.  These  metrics  provide  the  capability  to  detect 
and  report  data  defects  -  data  that  fails  to  meet  business  or  technical  requirements.  Data  quality 
metrics  use  statistical  methods  and  rule- based  techniques.  A  metric  is  a  process  or  algorithm 
that  may  involve  statistical  sampling,  mathematical  computations,  and  rule-based  infercncing 
("Zero  Defect  Data  Workbook:  Conducting  A  Data  Quality  Baseline  Audit",  reference  (n)). 
Applying  the  metric  to  a  DoD  data  sample  produces  measurements  that  will  indicate  the  presence 
and  level  of  data  defects  within  the  sample. 


G.  TOTAL  QUALITY  MANAGEMENT 


1.  Total  Quality  Management  (TQM)  is  both  a  philosophy  and  a  set  of  guiding  principles 
and  practices  that  represent  the  foundation  of  a  continuously  improving  organization.  It  applies 
human  resources  and  quantitative  methods  to  improve  the  materials  and  services  supplied  to  an 
organization,  all  the  processes  within  an  organization,  and  the  degree  to  which  the  needs  of  the 
customer  are  met  now  and  in  the  future.  It  integrates  fundamental  management  techniques, 
existing  improvement  efforts,  and  technical  tools  in  a  disciplined  and  focused  continuous 
improvement  process. 
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2.  TQM  addresses  the  quality  of  management  as  well  as  the  management  of  quality.  It 
involves  everyone  in  an  organization  in  a  systematic  long-term  endeavor  to  develop  processes 
that  are  customer  oriented,  flexible  and  responsive,  and  constantly  improving  on  quality. 
Quality  includes  any  factor  of  product  or  service  of  value  to  a  customer.  Ultimately,  TQM  is 
a  means  through  which  an  organization  creates  and  sustains  a  culture  committed  to  continuous 

improvement. 


H.  COSTS  OF  DATA  QUALITY 


1.  From  an  engineering  perspective,  cost  of  quality  can  also  be  viewed  as  the  losses  that 
are  caused  by  a  product's  functional  parameters  deviating  from  its  desired  target  value.  A 
significant  point  is  that  the  cost  of  quality  increases,  not  only  when  the  product  is  out  of 
specifications,  but  also  when  the  product  falls  within  specifications  but  deviates  from  target 
values.  With  regard  to  data,  rule-sets  and  metrics  measure  the  quality  of  data.  The  cost  of 
quality  depends  on  the  cost  of  proactively  incorporating  quality  into  the  DoD  data  process, 
versus  the  cost  of  failure  to  build  quality  into  the  DAdm  program. 


2,  As  defined  in  the  "Data  Quality  Engineering  Study",  (reference  (f)),  the  cost  of  quality 
can  be  classified  into  four  main  categories:  prevention  costs,  appraisal  costs,  internal  failure 
costs,  ;  i  external  failure  costs.  The  first  two  categories  refer  to  the  costs  of  incorporating 
quality,  known  as  the  costs  of  control  and  the  last  two  refer  to  the  costs  of  not  meeting  quality 
standards,  known  as  the  costs  of  failure  of  control. 


a.  Prevention  costs  are  the  costs  of  preventing  data  quality  problems  from 
happening.  Included  in  this  category  are  the  costs  of  setting  up  a  quality  engineering 
infrastructure,  training  employees  about  quality,  planning  for  quality  and  establishing  methods 
and  procedures  for  adding  quality  to  data. 


I 


b.  Appraisal  costs  are  the  costs  of  auditing  and  evaluating  data  quality.  Money 
will  be  spent  to  inspect,  test,  and  review  data  against  quality  requirements. 

c.  Internal  failure  costs  are  the  costs  to  the  organization  responsible  for  poor 
quality.  This  category  includes  the  costs  of  workarounds,  redesign  and  rework. 
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d.  External  failure  costs  are  the  costs  of  poor  data  quality  to  the  usen.  If  the  data 
error  rate  becomes  unacceptably  high,  usen  may  quit  using  the  system  and  look  for  other  dau 
sources  (including  other  ISs),  try  to  manually  collect  dau  which  already  exists  in  the  system, 
or  create  their  own  homegrown  manual  systems. 

3.  Problems  with  dau  quality  result  in  a  wide  range  of  costs  to  the  usen.  The  following 
describe  a  few  examples  based  on  the  categories  of  costs  of  quality; 

a.  Lost  Business  -  Inaccurate  dau  in  airline  reservation  systems  cause  planes  to 
take  off  oniy  one-third  to  one-half  full,  costing  airlines  millions  of  dollars.  A  large  New  York 
securities  broker  misses  a  big  trading  opportunity  when  incomplete  dau  is  enured  into  a  new 
risk  management  systems  and  loses  more  than  $200  million. 


b.  Lost  Production  -  Unreliable  project  management  dau  cause  aerospace  firms 
and  other  government  contractors  to  have  cost  overruns  and  lau  product  deliveries.  Unreliable 
production  management  dau  cause  manufacturers  to  rework  products  not  meeting  quality 
standards. 


c.  Lost, Assets  -  A  dau  entry  error  causes  an  insurance  company  to  issue  a 
monthly  check  for  $32,000  instead  of  the  $32  to  which  the  beneficiary  was  entitled.  Incorrect 
billings  cause  utilities,  telephone,  and  transp nation  companies  to  lose  money. 


d.  Legal  Liability  -  Inaccurate  spousal  information  and  home  addresses  in  human 
resource  ISs  may  prevent  compliance  with  the  Consolidated  Omnibus  Budget  Reconciliation  Act 
(COBRA).  COBRA  specifies  that  terminated  employees  are  entitled  to  benefits  coverage  six 
months  after  they  leave  the  company.  Invalid  warrants  and  old  warrants  in  the  National  Crime 
Information  Center  (NCIC)  Wanted-Persons  File  place  people  at  risk  of  being  falsely  detained 
and  arrested. 
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TOTAL  DATA  QUALITY  MANAGEMENT  PROCESS 


A.  INTRODUCTION 


1.  DoD  DAdm  must  ensure  that  DoD  operations  and  decision  making  are  supported  with 
secure  data  that  meets  needs  in  terms  of  availability,  accuracy,  timeliness,  and  integrity. 
Therefore,  DAds  throughout  DoD  must  manage  the  quality  of  data  throughout  the  data  life-cycle. 
To  support  these  efforts,  DoD  DAdm  has  established  a  Total  Data  Quality  Management  (TDQM) 
process  which  shall  be  used  to  ensure  DoD  data  quality. 


B.  DoD  TOTAL  DATA  QUAIITY  MANAGEMENT. PROCESS 


1.  The  DoD  Total  Data  Quality  Management  (TDQM)  process  encompasses  the 
processes,  methodologies,  tools  and  techniques  from  the  Total  Quality  Management  (TQM)  and 
Data  Quality  disciplines.  TDQM  applies  the  principals  of  TQM  as  described  in  DoD  5000.51- 
G,  "DoD  Total  Quality  Management  Guide"  (reference  (o))  to  the  management  of  data.  The 
five  (5)  steps  of  the  DoD  TDQM  process  are: 


Step  1:  Establish  the  Data  Quality  Assurance  Environment 

Step  2:  Scope  Data  Quality  Projects  and  Develop  Implementation  Plans 

Step  3:  Implement  Data  Quality  Projects  Using  The  Data  Quality  Engineering 

(DQE)  Methodology 

Step  4:  Evaluate  Data  Quality  Assurance  Progress 

Step  5:  Review,  Approve,  and  Implement  Data  Quality  Assurance 

Recommendations 
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2.  This  Chapter  describes  the  five  TDQM  steps  used  to  perform  DoD  data  quality 
assurance.  Chapter  4  introduces  the  DcD  Data  Quality  Engineering  (DQE)  methodology  that 
supports  the  DoD  TDQM  Process.  Chapter  5  describes  the  tools  and  techniques  used  in  TDQM 
and  DQE. 


C.  STTP  1;  ESTABLISH  THE  DATA  QUALITY. ASSURANCE  ENVIRONMENT 


1.  In  this  step,  the  management  and  cultural  environments  for  data  quality  assurance  are 
established.  The  TDQM  process  is  a  total  organizational  approach  toward  the  continuous 
improvement  of  data.  It  requires  DoD  functional  management  to  exercise  the  leadership 
necessary  to  establish  the  conditions  for  the  data  quality  assurance  to  flourish. 


2.  By  creating  a  constancy  of  purpose,  a  common  direction  for  all  organizational  elements 
is  established  and  ensures  that  efforts  at  all  levels  contribute  to  achieving  broad  objectives 
relevant  to  the  entire  organization.  Communicating  the  organization’s  data  quality  goals  and 
objectives  throughout  the  organization  is  essential  to  focusing  on  improvement  efforts. 


Establishing  the  data  quality  environment  consists  of: 


a.  Performing  Strategic  Planning  for  Data  Quality  Assurance,  and 

b.  Developing  the  Management  and  Cultural  Environments. 


3.  Performing  Strategic  Planning  for  Data  Quality  Assurance 


a.  The  DoD  Data  Administration  Strategic  Plan  (DASP)  (reference  (c))  provides 
the  comprehensive  and  long-term  direction  necessary  to  define,  plan,  implement,  and  operate 
the  DoD  Data  Administration  Program.  The  DASP,  as  documented  annually,  ensures  that  data 
products  will  be  managed  throughout  the  life  cycles  to  improve  business  methods,  efficiency  of 
operations,  and  the  quality  of  data.  The  DoD  DAd  is  responsible  for  developing  the  long-term 
vision,  mission,  and  goals  for  the  DoD  Data  Administration  Program.  The  DoD  DAd  also 
establishes  the  overall  goals  and  objectives  for  data  quality  assurance. 
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b.  Each  Functional  Data  Adminismtor  (FDAd)  and  Component  Data 
Administrator  (CDAd)  prepares  a  strategic  plan  for  their  respective  Functional  Area  or 
Component  in  accordance  with  the  annual  planning  guidance  provided  by  the  DoD  DAd.  The 
Functional  Area  and  Component  DASPs  must  include  the  operational  goals,  objectives,  and 
description  of  tasks  for  data  quality  activities.  Throughout  DoD,  DAds  and  DBAds  must 
incorporate  their  functional  area  or  component  data  quality  goals  and  objectives  into  their 
organization's  annual  plans.  If  required,  the  DoD  DAd,  FDAds,  or  CDAds  will  provide 
guidance  and  assistance  in  the  development  of  these  plans. 


c.  By  assessing  the  vision  of  the  organization's  future  and  coming  to  a  consensus 
on  a  data  quality  goal,  the  FDAd,  or  CDAd  will  solidify  their  understanding  of  the  ultimate  data 
quality  goals.  They  must  communicate  that  goal  throughout  the  organization.  The  data  quality 
assurance  goal  should  be  in  concert  with  the  organization's  mission,  the  long-term  vision  of  the 
functional  area,  and  DoD  DAdm.  Realistic  strategies  to  obtain  these  goals  must  be  established. 


d.  Set  Measurable  Objectives.  The  data  quality  assurance  objectives  should  be 
both  measurable  and  quantifiable.  The  objectives  must  reflect  the  plans  for  the  data  migration 
strategy  for  the  functional  area  and  be  in  concert  with  the  overall  DoD  DAdm  objectives.  The 
first  objective  should  be  to  establish  initial  baseline  assessments  for  the  costs  and  quality  of 
legacy  and  migration  data.  This  initial  baseline  is  used  to  identify  significant  improvement  areas 
in  which  success  is  essential  to  DoD. 


e.  Develop  Data  Quality  Assurance  Action  Plans.  As  part  of  the  DASP,  action 
plans  should  be  developed  for  obtaining  the  objectives.  The  action  plan  should  describe  a 
strategy  to  meet  the  data  quality  goal,  and  should  describe  the  organization’s  resources,  tasks 
and  milestones  needed  to  implement  the  data  quality  objectives  identified  in  the  DASP  (reference 
(c)).  Resource  requirements  should  also  be  provided  for  each  action  plan  in  accordance  with 
the  DASP  guidance. 


4.  Developing  The  Management  and  Cultural  Environment 


a.  Establish  Who  Is  Ultimately  Responsible.  To  create  a  successful  data  quality 
program,  it  is  vital  to  determine  the  individual  who  is  ultimately  responsible  for  setting  policy 
and  for  providing  top-management  functional  support.  In  most  cases  in  DoD,  this  would  be  the 
Functional  Activity  Program  Manager  (FAPM)  or  the  Office  of  the  Secretary  of  Defense, 
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Principal  Staff  Assistant  (OSD  PSA).  However,  depending  on  the  organizational  structure,  look 
within  line  management  and  identify  the  functional  manager  who  approves  the  organization  s 
annual  plan.  This  individual  is  critical  to  developing  the  data  quality  management  and  cultural 
environment. 


b.  Help  The  Functional  Manager  More  Aware.  When  beginning  to 

develop  ideas  for  the  initial  data  quality  improvement  effort,  the  first  step  for  the  DAd  is  to 
make  sure  that  the  functional  manager  is  aware  of  the  DoD  IM  initiatives  and  data  quality 
concerns.  Building  management  awareness  of  the  need  for  change  and  the  benefits  of  data 
quality  improvements  by  the  TDQM  process,  will  help  establish  a  foundation  for  continuous  data 
improvement  efforts.  Awareness  can  be  increased  by  providing  information  on  the  TDQM 
process,  advocating  the  manager's  attendance  at  DoD  DAdm  conferences  on  data  quality 
assurance,  and  inviting  his/her  participation  in  TDQM  activities.  Training  and  seminars  offered 
by  recognized  leaders  in  data  quality  assurance  can  also  be  instrumental  in  building  awareness. 
Management  should  be  made  aware  of  DoD  policies  and  procedures  for  data  quality  assurance 
through  the  review  of  DoD  Directives  and  Instructions,  and  Procedure  Manuals,  The  need  for 
data  quality  assurance  activities  may  also  be  demonstrated  by  citing  specific  incidences  when 
poor  quality  has  impacted  performance  within  the  organization. 


c.  Obtain  Management  Support  •  For  any  data  quality  assurance  effort  to  be 
successful,  management  must  support  and  approve  the  organization's  overall  plan  for  data 
quality.  Top-management  can  send  signals  or  exhibit  behavior  that  can  either  greatly  enhance 
data  quality  assurance  progress  or  unacceptably  restrict  it.  By  obtaining  management  support, 
the  potential  for  data  quality  improvements  are  great.  Through  encouraged  manager's  support, 
there  is  an  increased  potential  to  move  the  commitment  to  data  quality  assurance  throughout  the 
organization. 


d.  Prototype  Effort.  In  order  to  obtain  top-management  support,  the  DAd  and 
other  functional  experts  may  conduct  a  small,  prototype  effort.  This  effort  should  be  used  to 
make  a  case  for  the  ramifications  of  having  poor  quality  data  and  the  costs  associated  with  poor 
data.  These  assessments  should  be  presented  to  top-management  with  a  supporting  Data  Quality 
Plan.  When  selecting  a  prototype  effort,  choose  familiar  data  that  will  not  interfere  with  the 
ongoing  business  activities  of  the  organization's  current  assignments.  Choose  data  with  low  risk 
and  low  visibility,  but  that  is  critical  to  the  organization's  success.  From  these  efforts,  show 
the  results  of  the  data  quality  analysis  and  present  overall  cost  estimates  for  poor  data  quality. 
From  this  prototype  effort,  attempt  to  gain  management  understanding  and  support  to  implement 
a  TDQM  program  in  the  organization.  Procedures  for  conducting  a  prototype  effort  are 
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discussed  in  deoil  under  Step  2  below.  Once  support  has  been  established,  the  organization 
should  begin  to  esobiish  the  data  quality  environment. 


e.  Create  An  Environment  In  Which  Continuous  Data  Improvement  Is  A  Wav 
Of  Life.  TDQM  concentrates  on  providing  quality  data  that  consistently  meets  the  needs  and 
expectations  of  the  user.  Every  member  of  DoD  must  know  the  purpose  of  his/her  job,  his/her 
customer(s),  and  his/her  relation  to  others  within  DoD  for  providing  end-user  satisfaction. 
Everyone  must  know  the  data  users  requirements.  The  data  quality  objectives  of  each  functional 
area  of  DoD  must  reflect  a  perspective  that,  when  combined  with  other  areas  of  DoD,  will 
provide  the  synergy  that  produces  "Quality  Information  for  a  Strong  Defense". 


(1)  All  DoD  personnel  must  make  continuous  data  improvement  a  part  of 
the  daily  routine,  whereby  it  is  integrated  into  all  aspects  of  work.  Continuous  data 
improvement  only  approaches  maturity  when  it  is  applied  routinely  to  all  of  the  organization's 
efforts.  Routine  application  entails  ensuring  data  quality  throughout  the  data  life-cycle  including 
collecting  and  using  data.  DAds  and  functional  managers  should  also  routinely  assess  process 
suitability  and  remove  roadblocks  to  improvement  efforts. 


f.  To  build  a  cultural  environment  in  which  data  quality  becomes  everyone's 
responsibility,  the  following  strategies  should  be  implemented:  (1)  Demonstrate  Leadership,  (2) 
Build  Awareness,  (3)  Create  Open  Lines  of  Communication,  and  (4)  Develop  Teamwork. 


(1)  Demonstrate  Leadership.  A  leader  needs  to  be  established  for  the  data 
quality  function.  This  individual  will  provide  the  lead  for  establishing  baselines,  performing 
assessments,  prioritizing,  providing  solutions,  and  making  final  improvement  recommendations. 
This  role  may  be  performed  by  the  DBAd,  DAd,  AIS  project  manager,  or  technical  developers. 
This  individual  has  the  overall  operational  responsibility  for  the  quality  of  the  data.  This 
individual  leader  should  be  aware  of  the  TQM  discipline  and  DAdm.  This  leader  will  help  top- 
management  establish  the  cultural  environment  for  implementing  TDQM.  The  leader  establishes 
a  formal  or  informal  DQE  team  consisting  of  the  DAd,  DBAd  and  other  technical  and  functional 
experts  and  has  the  overall  responsibility  for  implementing  the  DoD  DQE  methodology. 


(2)  Build  Awareness.  Building  awareness  (understanding  what  data  quality 
assurance  is  and  why  it  is  important  to  the  organization)  is  perhaps  one  of  the  most  important 
steps  in  implementing  TDQM.  The  organizatio  must  become  aware  of  the  need  to  improve  the 
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quality  of  data  and  understand  the  various  tools  and  methodologies  available  for  improvement 
efforts. 


(a)  The  development  of  DoD  procedures  provides  an  opportunity 
to  make  the  organization  more  aware  of  TDQM  efforts.  As  discussed  above,  it  is  also  important 
to  help  the  functional  manager  be  more  aware  (through  annual  reports  and  assessment  reports). 
DAdm  strategic  planning  should  plan  for  data  quality  assurance  in  the  future.  Everyone  should 
become  more  aware  of  TDQM  through  training,  seminars,  and  newsletters.  Information  should 
be  shared  (in  terms  of  both  progress  and  problems)  across  functional  and  component  areas.  The 
DoD  DAd  will  publish  annual  reports  on  the  progress  of  DoD  data  quality  assurance. 


(3)  Create  Open  Lines  of  Communication.  As  the  awareness  of  data 
quality  assurance  is  being  raised  throughout  the  organization,  begin  to  establish  open  lines  of 
communication  both  horizontally  and  vertically.  Open  communication  permits  teams  to  work 
through  problems,  overcome  barriers,  and  find  encouragement  and  support  from  other  data 
quality  assurance  efforts. 


(a)  Esublish  a  process  where  data  users  can  freely  report  problems 
with  data,  and  can  obtain  quick  resolution.  Within  the  DoD  organization,  develop  a  monitoring 
and  tracking  process  to  allow  for  user  feedback.  These  communication  processes  need  to  be 
developed  with  the  organization's  operating  procedures  to  create  an  atmosphere  that  facilitates 
information  sharing. 


(4)  Develop  Teamwork.  Teamwork  is  the  engine  that  drives  many 
improvement  efforts.  Creating  teams  allow  the  application  of  diverse  skills  and  experience  to 
problem  solving.  An  atmosphere  of  teamwork  should  permeate  the  organization,  affecting  not 
only  formal  team  efforts  but  also  each  individual's  interaction  in  the  organization.  Within 
functional  areas,  performance  improvement  teams  provide  cross-functional  orientation,  and  the 
employees  on  these  teams  become  involved  in  process  issues.  Thus,  the  entire  DoD  is 
effectively  interlinked  to  form  an  ideal  performance  improvement  setting. 


(a)  DoD  must  implement  the  TDQM  program  using  a  structured 
teamwork  approach,  establishing  DAdm  work  groups  similar  to  TQM  project  action  teams. 
These  work  groups  are  made  up  of  DAds,  DBAds,  technical  developers,  program  managers 
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(PM),  and  functional  experts.  These  work  groups  are  used  to  identify  root  causes  and  to 
develop  recommendations  for  data  quality  improvements. 


D.  STEP  2:  IDENTIFY  DATA  QUALITY _ PROJECT _ AHQ - PEYELQE 

IMPLEMENTATION  PLAN 


1.  The  next  step  in  the  TDQM  process  is  to  identify  a  data  quality  improvement  project. 
This  data  quality  improvement  project  may  be  the  organization's  initial  data  quality  effort,  or 
may  be  a  prototype  effort  to  gain  top  management  support.  The  DoD  DAd  will  provide  the 
guidance  and  consultation  to  support  Steps  1  and  2  in  a  DoD  TDQM  effort.  DoD  DAdm 
establishes  the  overall  DoD  data  quality  assurance  environment  and  will  assist  organizations  in 
identifying  improvement  projects,  and  developing  implementation  plans  and  prototype  efforts. 


2.  Choose  Early  Efforts  in  Visible  Areas  Critical  To  Success  The  success  or  failure  of 
initial  TDQM  efforts  and  projects  can  greatly  affect  how  easily  the  organization  adopts  TDQM 
ideas.  Select  projects  that  (1)  have  a  high  chance  of  success,  (2)  have  the  highest  external  costs 
and,  (3)  where  the  greatest  improvements  can  be  made.  Addressing  the  critical  issues  and 
examining  historical  data  problems  first,  increases  the  chance  that  the  results  will  increase  the 
attractiveness  of  TDQM  to  top  management. 


3.  Prototype  Effort.  If  there  is  not  top  management  support  for  data  quality  efforts, 
perform  a  prototype  effort,  choosing  a  systems  with  low  risk,  low  visibility,  but  that  is  critical 
to  the  organization's  success.  Focus  on  familiar  data  where  individual  expertise  can  be 
provided.  Select  an  initial  effort  that  is  neither  too  large  that  it  is  doomed  for  failure  from  the 
start,  or  so  small  that  improvements  will  essentially  go  unnoticed. 


4.  Identifying  The  Data  Quality  Project.  When  identifying  a  data  quality  project,  many 
criteria  should  be  reviewed  to  determine  the  suitability  of  the  project.  Prior  to  the  evaluation 
and  selection  of  data  for  data  quality  improvement  (whether  an  initial  effort  or  a  prototype  effort) 
a  set  of  selection  criteria  must  be  developed.  Major  evaluation  areas  should  be  identified  and 
selection  criteria  are  to  be  developed  for  each  area.  The  evaluation  areas  should  focus  on  the 
availability  of  information  needed  for  the  DoD  Data  Quality  Engineering  (DQE)  process  detailed 
in  Step  3.  Criteria  and  weighting  factors  should  be  applied  to  evaluate,  select,  and  perform  the 
DoD  DQE  analysis.  Chapter  5  provides  a  technique  with  specific  evaluation  factors  for 
conducting  the  Data  Selection  Evaluation. 
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5.  Once  the  data  has  been  selected  for  the  TDQM  effort,  project  scope  must  be 
determined,  including  data  quality  metrics,  and  a  specific  implementation  plan  must  be 
developed.  For  either  an  initial  or  prototype  effort,  the  project  should  be  developed  in 
accordance  with  the  goals  and  objectives  identified  within  Step  1.  This  plan  should  detail  the 
methods  and  tools  required  to  (1)  Minimize  DoD  data  defects  in  the  selected  data  sample,  and 
(2)  Reduce  DoD  data  management  costs. 


a.  Implementation  Plan.  Once  the  data  has  been  selected,  an  implementation 
plan,  to  integrate  DQE  into  the  data  quality  project,  must  be  developed.  This  will  include 
identifying  the  processes  and  procedures  required  to  operate  DQE  and  to  analyze  and  resolve 
identified  data  errors,  and  identifying  how  DQE  will  be  used  by  the  various  components  of  the 
DoD  DAdm  infrastructure. 


(1)  Develop  a  detailed  Plan  of  Action  and  Milestones  for  the  application 
of  DQE  to  the  selected  data  elements.  This  will  include  identifying  and  prioritizing  the  detailed 
tasks,  and  subtasks,  required  to  implement  the  DQE.  From  initial  analysis,  identify  the  target 
database,  and  obtain  the  appropriate  information  on  the  data  structure,  content,  support 
organization  and  other  required  details.  Also,  define  the  objective  metrics  designed  to  measure 
the  continuous  improvement  of  the  quality  of  the  metadata  as  a  result  of  the  DQE 
implementation. 


(2)  Define  the  methods  and  tools  to  be  used.  Establish  the  required  costs 
associated  with  acquiring  these  tools.  Also  define  the  techniques  for  evaluating  and  utilizing  the 
tools  for  capturing  data  quality  business  rules,  measuring  the  data  and  providing  statistical 
reports  using  metrics. 


(3)  Finally,  an  overall  project  schedule  and  funding  schedule  should  be 
presented.  These  schedules  will  define  actual  deliverables  and  target  dates  of  completion.  An 
overall  resource  funding  schedule  should  also  be  prepared  and  presented. 


6.  Step  1  and  Step  2  provide  the  functional  infrastructure  to  begin  to  implement  a  TDQM 
project.  Step  3  Implement  Data  Quality  Projects  Using  The  Data  Quality  Engineering  (POE! 
MsitlttdPlggy  provides  a  technical,  structured  approach  for  data  quality  improvement  efforts. 
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1.  The  analysis  and  improvement  of  data  relies  on  a  structured  approach,  such  as  the 
DoD  Data  Quality  "Engineering  (DQE)  methodology.  The  DoD  DQE  process  consists  of  a 
methodology  supplemented  by  an  automated  tool  to  clearly  define  the  source  of  persistent 
problems  with  data  and  database  systems.  The  DQE  process  includes  precise  techniques  for 
selecting  the  best  corrective  actions  and  monitoring  the  progress  towards  achieving  data  quality 
objectives.  The  overall  objectives  of  the  DQE  methodology  are  to  assess  and  validate  specific 
problems,  identify  root  causes,  and  improve  the  quality  of  the  data.  In  order  to  meet  these 
objectives,  the  DQE  methodology  first  determines  the  exact  nature  and  magnitude  of  the  problem 
base.  DQE  analyses  focus  on  the  logical  data  structure,  data  element  definitions,  and  data  values 
(the  actual  "instances"  of  data).  Further,  the  DQE  methodology  supports  the  selection  of  a 
quantifiable  "measure  of  success"  and  supports  iterative  assessments  of  progress  toward  the 
objective.  The  methodology  does  not  stop  with  the  problem  definition  and  identification  of 
cause,  it  also  points  the  analyst  toward  (or  "triggers")  a  technique  (e.g.,  dau  modeling,  process 
modeling,  data  standardization,  policy  or  procedure  modifications,  training  requirements)  that 
will  correct  the  problem.  In  short,  the  DQE  methodology  identifies  problems  and  root  causes, 
triggers  an  appropriate,  cost  effective,  focused,  improvement  process,  and  provides  a  means  of 
measuring  success  ("Data  Quality  Engineering  (DQE)  Pilot  Project,  Technical  Report,  reference 
(P)). 


2.  In  DoD,  the  DQE  methodology  is  used  is  to  implement  a  data  quality  assessment 
project,  identified  in  Step  2  of  the  TDQM  process,  or  to  investigate  an  end  user's  observation 
of  a  specific  data  quality  problem.  The  data  quality  assurance  leader  identified  in  Step  1  of  the 
TDQM  process  has  the  overall  responsibility  for  implementing  the  DoD  DQE  methodology. 
The  leader  establishes  a  formal  or  informal  DQE  team  consisting  of  the  DAd,  DBAd  and  other 
technical  and  functional  experts.  Throughout  the  DoD  DQE  effort,  the  leader  meets  periodically 
with  team  members  to  exchange  ideas,  review  interim  findings,  seek  answers  to  specific 
questions,  and  resolve  issues.  The  results  of  a  DoD  DQE  effort  are  documented  in  a  Data 
Quality  Baseline  Assessment  Report.  The  procedures  for  developing  a  Data  Quality  Baseline 
Assessment  Report  are  presented  in  Chapter  4  of  this  Manual.  The  report  not  only  describes 
the  current  baseline  for  the  quality  of  data,  but  also  the  baseline  of  the  associated  poor  quality 
costs. 
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3.  The  DoD  DQE  methodology  is  comprised  of  four  (4)  major  activities:  DEFINE. 
MEASURE,  ANALYZE,  and  IMPROVE.  Chapter  4  describes  the  complete  DoD  DQE 
methodology  in  detail.  Chapter  5  introduces  the  tools  and  techniques  to  implement  the  DoD 
DQE  methodology. 


STEP  4:  EVALUATE  DATA  QUALITY  ASSURANCE  PROGRESS 


1.  Measurement,  evaluation,  and  reporting  are  essential  elements  of  the  TDQM  process. 
These  elements  focus  on  the  effectiveness  of  improvement  efforts  and  identify  areas  for  future 
improvement  efforts.  Information  on  TDQM  efforts  should  be  forwarded  by  the  DQE  team 
leader  to  the  DoD  DAd  so  that  it  can  be  included  in  annual  reports  and  included  in  future 
strategic  plans. 


a.  Implementation  Plan  Performance  Data.  A  basic  need  in  all  improvement 
efforts  is  the  ability  to  measure  the  value  of  the  improvement  in  units  that  are  pertinent  and 
meaningful  to  the  specific  tasks,  as  identified  in  the  project  Implementation  Plan.  Improvement 
is  evaluated  on  the  factors  of  both  performance  and  cost. 


(1)  Daa_QualiOL,Measurcs.  Using  established  data  quality  metrics 
(procedures  in  Chapter  4)  will  measure  data  performance  against  target  goals  for  data  quality. 
These  metrics  will  detect  and  report  data  defects  -  data  that  fails  to  meet  business  or  technical 
requirements.  Applying  the  metric  to  a  DoD  data  sample  produces  measurements  indicates  the 
presence  and  level  of  data  defects  within  the  sample. 


b.  Baseline  Assessment  Data.  The  DoD  DQE  methodology  provides  the  DAd 
with  a  means  for  identifying  and  assigning  responsibility  for  corrective  actions.  A  data  quality 
baseline  is  always  established  in  the  initial  DoD  DQE  effort.  The  DAd  can  then  assess  progress 
toward  achieving  data  quality  by  conducting  periodic,  identically  configured  DoD  DQE 
evaluations  on  the  database.  This  provides  a  comprehensive  indication  of  compliance  with  the 
quality  specifications  over  a  specified  time  period. 


2.  As  part  of  the  information  forwarded  for  the  annual  reports,  the  following  should  be 
included  in  a  section  on  accomplishments,  or  in  a  summary  report:  (1)  Identify  The  System 
Selected,  (2)  Identify  The  Number  of  Data  Elements  Assessed,  (3)  Identify  The  Number  of  Data 
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Errors  Detected,  (4)  Identify  The  Number  of  Data  Errors  Corrected,  and  (5)  Discuss  The 
Overall  Data  Quality  Improvements  Made. 


3.  Evaluate  Cultural  Environment.  In  this  stage,  all  procedures  for  data  quality  assurance 
should  be  re-evaluated.  The  overall  DASP  should  be  reviewed  and  updated  accordingly.  With 
regard  to  cost,  there  must  be  a  determination  of  what  is  the  acceptable  percentage  of  defect  data 
(target  parameters)  versus  the  cost  of  obtaining  zero  data  defects. 


G.  STEE^BEOESL  APPROVE.  AND  IMPLEMENT  DATA  QUALITY  ASSURANCE 

RECOMMENDATIONS 


1.  Within  Step  5,  review  all  actions  taken  an  determine  which  areas  can  be  improved 
upon.  All  DoD  employees  will  need  to  review  progress  with  respect  to  improvement  efforts  and 
modify  or  rejuvenate  existing  approaches  to  data  quality  assurance  for  the  next  progression  of 
methods.  This  constant  evolution  reinforces  the  idea  that  TDQM  is  not  a  program,  but  rather 
a  new  day-to-day  behavior  for  the  entire  DoD. 


2.  A  DAdm  performance  improvement  work  group  should  develop  measures  that  are 
appropriate  for  re-evaluating  their  TDQM  process.  This  would  include  reviewing  cost  baselines 
and  performance  results.  The  DoD  DAd  should  also  review  these  performance  results  against 
the  overall  goals  and  objectives  for  data  quality  assurance. 


3.  As  part  of  the  review  process,  action  plans  should  be  reviewed  by  the  DoD  DAd, 
FDAds,  or  CDAds  for  the  achievement  of  data  quality  assurance  objectives.  These  action  plans 
should  have  met  the  data  quality  goal,  and  should  have  described  the  organization’s  resources, 
tasks  and  milestones  needed  to  implement  the  data  quality  objectives  identified  in  the  DASP 
(reference  (c)).  Resource  requirements  should  be  analyzed,  in  reference  to  each  action  plan,  to 
determine  areas  of  improvement.  Training  resources  must  be  made  available  to  support  the 
organization’s  continued  TDQM  improvement  efforts.  Training  plans  should  be  updated  to 
provide  an  immediate  opportunity  to  use  new  skills  developed  throughout  the  TDQM  project. 


4.  Data  quality  recommendations  may  focus  on  developing  and  executing  Functional 
Process  Improvement  (FPI)  initiatives  to  reduce  future  data  defects.  Any  system  and/or  process 
defects  found  as  a  result  of  the  DQE  effort  should  have  been  forwarded  to  the  FAPM  for 
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correction.  The  FAPM  should  also  identify  and  analyze  root  causes  of  data  defects,  identify 
opportunities  for  systems  and/or  process  improvements,  and  prepare  an  implementation  plan  for 
approval  in  accordance  with  the  FPI  procedures  described  in  DoD  8320. 1-M,  "Interim  Guidance 
on  Functional  Process  Improvement*  (reference  (q)).  DoD  DAdm  must  perform  quality  control 
activities  to  track  and  document  continuous  process  improvements  for  DoD  data  quality 
assurance. 
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nnD  DATA  QUAT  TTV  ENGINEERING  METBOPQLQftY 


A.  INTRODUCTION 


1.  The  DoD  methodology  introduced  in  this  Chapter  was  derived  from  a  proven  U.S. 
Marine  Corps  Data  Quality  Engineering  (DQE)  methodology.  Appendix  A  presents  the 
application  of  the  DQE  methodology  to  the  DoD  data  environment  (the  U.S.  Marine  Corps). 
Through  technical  research,  evaluation  and  prototyping  activities,  it  has  evolved  into  a  standard 
DoD  DQE  methodology.  This  DoD  DQE  methodology  supports  the  Total  Data  Quality 
Management  (TDQM)  process  that  Chapter  3  identifies. 


B.  DoD  DATA  QUALITY  ENGINEERING  METHODOLOGY 


1.  The  following  sections  describe  the  DoD  DQE  methodology.  This  methodology  is 
based  on  the  Total  Quality  Management  philosophy,  the  U.S.  Marine  Corps  DQE  methodology, 
and  encompasses  research  and  evaluation  of  other  data  quality  methods,  tools  and  techniques. 
The  Data  Quality  Engineering  methodology  (supplemented  by  tools  and  techniques)  clearly 
defines  the  source  of  persistent  problems  with  data  and  database  systems.  The  DQE  process  also 
includes  precise  techniques  for  selecting  the  best  corrective  actions  and  for  monitoring  the 
progress  toward  achieving  data  quality  goals.  The  success  of  the  DQE  methodology  rests  in  the 
efficiency  of  an  innovative,  "data -centered'  approach.  The  DQE  methodology  allows  analysts 
to  achieve  detailed,  quantified  definitions  of  data  problems  ("Data  Quality  Engineering  (DQE) 
Pilot  Project,  Technical  Report",  reference  (p)). 


2.  The  overall  objectives  of  the  DQE  methodology  are  to  assess  and  validate  specific 
problems,  identify  root  causes,  and  improve  the  quality  of  the  data.  In  order  to  meet  these 
objectives,  the  DQE  methodology  first  determines  the  exact  nature  and  magnitude  of  the  problem 
base.  DQE  analyses  focus  on  the  logical  data  structure,  data  element  definitions,  and  data  values 
(the  actual  "instances"  of  data).  Further,  the  DQE  methodology  supports  the  selection  of  a 
quantifiable  "measure  of  success"  and  supports  iterative  assessments  of  progress  toward  the 
objective.  The  methodology  does  not  stop  with  the  problem  definition  and  identification  of 
cause,  it  also  points  the  analyst  toward  (or  "triggers")  a  technique  (e.g.,  data  modeling,  process 
modeling,  data  standardization,  policy  or  procedure  modifications,  training  requirements)  that 
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will  correct  the  problem.  In  short,  the  DQE  methodology  identifies  problems  and  root  causes, 
triggers  an  appropriate,  cost  effective,  focused,  improvement  process,  and  provides  a  means  of 
measuring  success  ("Data  Quality  Engineering  (DQE)  Pilot  Project,  Technical  Report", 
reference  (p)). 


3.  In  DoD,  the  DQE  methodology  is  used  is  to  implement  a  data  quality  assessment 
project,  identified  in  Step  2  of  the  TDQM  process,  or  to  investigate  an  end  user’s  observation 
of  a  specific  data  quality  problem.  The  data  quality  assurance  leader  identified  in  Step  1  of  the 
TDQM  process  has  the  overall  responsibility  for  implementing  the  DoD  DQE  methodology. 
This  leader  establishes  a  formal  or  informal  DQE  team  consisting  of  the  DAd,  DBAd  and  other 
technical  and  functional  experts.  Throughout  the  DoD  DQE  effort,  the  leader  meets  periodically 
with  team  members  to  exchange  ideas,  review  interim  findings,  seek  answers  to  specific 
questions,  and  resolve  issues.  The  results  of  a  DoD  DQE  effort  are  documented  in  a  Data 
Quality  Baseline  Assessment  Report.  The  report  not  only  describes  the  current  baseline  for  the 
quality  of  data,  but  also  the  baseline  of  the  associated  poor  quality  costs. 


4.  The  DoD  DQE  methodology  is  comprised  of  four  (4)  major  activities:  DEFINE, 
MEASURE,  ANALYZE,  and  IMPROVE.  The  DEFINE  activity  focuses  on  identifying  data 
quality  requirements  and  establishing  metrics.  The  MEASURE  and  ANALYZE  activities  focus 
on  measuring  and  assessing  the  data  quality,  identifying  root  causes  of  errors,  and  analyzing 
opportunities  for  improvement.  The  IMPROVE  activity  focuses  on  developing  and  executing 
improvement  initiatives  for  correcting  data  defects.  The  DoD  DQE  methodology  is  compatible 
with  Step  5  of  DoD  TQM  model  basic  performance  improvement  cycle  as  described  in  DoD 
5000. 1-G,  "DoD  Total  Quality  Management  Guide”,  (reference  (o)). 


5.  Figure  4-1  is  an  activity  model  that  depicts  the  DoD  DQE  methodology.  The  activity 
model  is  for  exposition  only  (FEO)  (i.e.  for  the  purposes  of  depicting  these  procedures).  The 
model  is  depicted  using  the  IDEFO  modeling  technique.  The  dashed  lines  represent  the  controls, 
mechanisms  and  outputs  that  exists  for  every  activity.  Guidance  are  the  directives,  instructions, 
or  procedures  that  describe  and  govern  the  roles  and  processes  for  implementing  the  DQE 
methodology.  Tool  Requirements  are  developed  as  a  result  of  attempting  to  perform  any 
activity.  The  Plan  Performance  Data  are  used  to  document  the  actual  resources  utilized, 
accomplished  and  objectives  achieved  as  a  result  of  performing  each  activity.  The  mechanisms, 
the  DQE  team,  Common  Tools  and  Source  Data  Experts,  are  actually  implemented  or  are  used 
to  perform  each  activity.  Common  Tools  are  described  in  Chapter  5  of  this  Manual.  The  DQE 
team  meets  regularly  throughout  the  DQE  process  and  are  considered  a  dedicated  resource  until 
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the  process  is  complete.  The  Source  Data  Experts,  although  not  on  the  DQE  team,  are  called 
upon  throughout  the  DQE  process  for  consultation  and  validation  during  the  DEFINE, 
MEASURE  and  ANALYZE  activities,  and  to  implement  corrective  actions  and  improvement 
opportunities  during  the  IMPROVE  activity.  The  following  sections  discuss  in  detail  each  of 
the  activities  of  the  DoD  DQE  methodology. 


C.  DEFINE 


1.  The  DEFINE  activity  focuses  on  identifying  data  quality  requirements  and  establishing 
metrics.  All  of  the  preliminary  activities  necessary  to  measure  and  assess  the  data  quality  are 
performed  as  pan  of  the  DEFINE  activity.  The  steps,  tools,  and  techniques  of  the  DEFINE 
activity  are  presented  in  Table  4-1  below. 


OBJECTIVES 

TOOLS/ 

TECHNIQUES 

Identify  data  quality 

1.  Scope  the  Problem 

DDRS 

requirements  and 

Data  Selection 

establish  metrics. 

2.  Identify  and  Review 

Data  Model 

Documentation 

Activity  Model 
NGT 

3.  Develop  Quality  Parameters 

Cost  of  Quality 
Control  Chart 

QFD 

DTR 

Pareto  Chart 

Histogram 

Table  4-1  DoD  DQE  DEFINE  Activity 
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2.  Scope  the  Problem. 

a.  In  the  first  step,  'Scope  the  Problem",  the  DQE  team  will  determine  the 
specific  data  quality  problems  for  the  assessment  and  gather  the  appropriate  data  experts  and 
supporting  information.  In  order  to  scope  the  problem,  the  DQE  team  must: 

(1)  Determine  the  Objective 

(2)  Identify  Data  Experts 

(3)  Identify  Historical  Data  Problems  (If  Available),  and 

(4)  Identify  Information  and  Processes  That  Need  To  Be  Addressed 


b.  Determine  the  Objective.  The  DQE  team  must  agree  on  a  clear  objective  for 
the  data  quality  engineering  effort.  -Agreeing  on  an  objective  helps  the  team  to  develop  a  shared 
focus,  provides  a  sense  of  direction,  eliminates  frustration  and  conflict  between  the  members, 
and  builds  trust  in  order  to  gain  acceptance.  This  is  important  because,  as  assessments  and 
analyses  are  performed,  functional  and  technical  experts  responsible  for  the  data  should  not  point 
blame  or  feel  threatened  because  of  suspect  data  errors  or  root  causes.  The  objective  is  also 
used  for  measuring  the  success  of  the  baseline  assessment  and  to  ensure  that  the  team  remains 
focused  on  the  scope  of  the  project  while  assessments  and  analyses  are  being  conducted.  The 
objective  should  include  the  type  of  data  that  the  DQE  team  plans  to  assess. 


c-  Identify  Data  Experts.  Once  the  objective  has  been  determined,  the  DQE  team 
should  identify  the  experts  (i.e.  end  users,  data  entry  clerks,  DBAds,  programmers)  that  have 
knowledge  of  the  how  the  data  values  are  created,  updated,  and  maintained.  These  data  experts 
should  also  understand  the  objective  and  be  available  for  consultation  throughout  the  DQE  effort. 


d.  Identify  Historical  Data  Problems.  To  prepare  for  the  development  of  quality 
parameters  for  the  assessments,  the  DQE  team  along  with  the  data  experts  should  identify 
specific  historical  data  problems  or  concerns  for  the  selected  type  of  data.  Data  quality 
requirements  should  be  based  on  business  and  end  user  requirements.  Business  requirements 
can  be  gathered  from  existing  policies,  procedures  or  other  documentation  corresponding  to  the 
data.  Some  techniques  that  would  be  useful  in  developing  a  set  of  end  user  requirements  are: 
interviewing,  brainstorming,  and  the  Nominal  Group  Technique  (NGT).  (Chapter  5  of  this 
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Manual  further  defines  these  tools  and  techniques.)  The  NGT  is  similar  to  brainstorming  and 
is  especially  effective  with  new  DQE  teams  or  data  experts  that  are  reluctant  to  disclose 
opinions,  ideas,  and  feelings.  If  several  problems  are  identified  the  business  and  end  user 
requirements  should  be  analyzed  to  further  scope  the  data  quality  problems.  If  the  DQE  effort 
is  pan  of  an  ongoing  data  quality  improvement  inidadve,  whereby  the  data  quality  baseline  is 
being  reassessed  or  reestablished,  then  previously  prepared  tools  such  as  the  Pareto  Chan, 
Histogram,  or  Checksheet  would  also  be  useful.  A  useful  tool  to  analyze  the  business  and  user 
requirements  is  the  Quality  Function  Deployment  (QFD).  The  QFD  tool  may  also  be  used  to 
establish  new  data  quality  requirements  in  order  to  expand  the  baseline  for  assessment.  The 
process  for  using  QFD  is: 


(1)  Categorize  the  end  user  needs  or  business  requirements  into  the  data 
quality  classifications  idendfied  in  Chapter  2  of  this  Manual,  (e.g.  accuracy,  completeness, 
consistency,  timeliness);  and 


(2)  Within  each  category  prioritize  the  requirements  in  order  of  importance 
or  impact  to  the  organization  ("Teams  for  Excellence:  Skills,  Strategies,  &  Implementation 
Exercise  Book,  reference  (r)). 


c-  Identify  Information  and  Processes  That  Need  to  Be  Addressed.  Once  the 
most  specific,  historical  data  quality  problems  have  been  identified,  then  the  DQE  team  should 
begin  to  identify  supporting  information  and  processes  for  the  data.  This  task  helps  the  team 
to  focus  in  on  the  documentation  and  systems  that  will  need  to  be  reviewed  in  the  second  step 
of  the  DEFINE  activity. 


2.  Identify  and  Review  Documentation. 


a.  The  overall  purpose  of  this  step  in  the  DEFINE  activity  is  to  identify  the 
applicable  metadata  and  physical  data  structures  that  will  be  necessary  to  develop  the  quality 
parameters.  An  extensive  amount  of  time  may  be  spent  on  this  step,  depending  on  the  amount 
of  effort  required  to  identify  and  gather  the  necessary  documentation.  In  many  cases,  due  to  the 
lack  of  configuration  control  applied  to  the  maintenance  of  documentation  for  legacy  systems, 
data,  and  application  code,  the  information  is  either  outdated  or  nonexistent.  In  some  cases, 
even  the  most  recent  documentation  is  not  current  or  understandable.  As  DoD  migration 
systems  are  reverse  engineered,  it  is  important  to  continuously  update  the  documentation  along 
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with  the  system  or  data  it  describes.  As  a  result,  a  technical  expert  (e.g.  data  analyst)  will  be 
able  to  perform  the  following  steps: 

(1)  Review  the  Defense  Data  Repository  System  (DDRS)  For  Standard 
(Candidate  or  Developmental  Data  Elements) 

(2)  Review  Logical  Data  Model 

(3)  Analyze  System  Or  Reverse  Engineering  Documentation 

(4)  Review  Policies  and  Regulations 

(5)  Assess  If  There  Is  Enough  Information  To  Proceed,  If  Not,  May 
Have  To  Do  Full  Or  Partial  Reverse  Engineering 


b.  Review  The  DDRS.  As  DoD  data  elements  are  standardized,  standard  data 
quality  parameters  (rule  sets  and  metrics)  will  be  made  available  for  assessing  data  quality. 
Therefore,  the  data  analyst  should  first  review  the  standard  data  elements  in  the  DDRS  for  the 
metadata  that  is  most  applicable  to  the  information.  If  a  standard  data  element  is  not  available, 
consider  using  a  candidate  or  developmental  data  element.  Also  review  the  data  elements  in  the 
system  or  organization  data  dictionary  for  the  availability  of  applicable  metadata. 

c.  Review  Logical  Data  Model.  If  a  data  element  is  identified  for  use  from  the 
DDRS,  then  the  DoD  Data  Model  or  associated  logical  data  model  should  be  reviewed. 
Otherwise,  functional  area/activity,  Component  or  organization  logical  data  models  should  be 
reviewed.  These  data  models  are  developed  during  the  Identification  and  Definition  phase  of 
the  Data  Life  Cycle  (see  Chapter  2  of  this  Manual)  either  through  top  down  data  modeling  or 
reverse  engineering  efforts  and  as  a  result  of  performing  a  Business  Process  Improvement 
Project  as  described  in  DoD  8020. 1-M  (reference  (q)). 


d.  Analyze  System  or  Reverse  Engineering  Documentation.  In  order  to  gain  a 
better  understanding  of  the  current  physical  data  structures,  the  data  elements,  and  the  business 
processes  that  the  system  and  data  support,  the  source  system  database  structure  should  be 
analyzed.  This  source  system  detailed  database  structure  review  begins  with  an  analysis  of  the 
data  element  metadata  instances,  the  database  specification,  data  and  process  models,  system 
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functional  description  documents,  and  system  design  specifications.  Baselining  the  current  data 
structures  and  the  available  documentation  lays  the  foundation  for  the  data  analysis  process. 


e.  Review  Policies  and  Regulations.  Review  policies  and  regulations  to  identify 
specific  information  that  would  be  applicable  to  the  data.  Such  information  may  consist  of 
associated  procedures,  allowable  values,  responsible  personnel,  and  timeliness  criteria.  It  is 
important  to  obtain  the  most  current  documents  and  be  aware  of  any  that  are  undergoing 
modifications. 


f.  Assess  If.There  is  Enough  Information  to  Prrw^  it  n0t  unusual  for  some 
or  all  of  these  documents  to  either  be  outdated  or  nonexistent.  Depending  on  the  necessity,  the 
information  required  will  be  derived  either  informally  or  formally.  Throughout  the  DEFINE 
activity  the  DQE  leader  needs  to  assess  if  enough  information  is  available  to  proceed  with  the 
DQE  effort.  If  not,  a  full  or  partial  reverse  engineering  project  may  have  to  be  conducted. 
The  DQE  methodology  does  not  require  a  full  reverse  engineering  effort,  but  overall  the  reverse 
engineering  products  will  provide  significant  input  into  the  DQE  effort  and  will  reduce  the  un- 
front  analysis  time.  When  conducting  a  reverse  engineering  effort  it  is  important  to  capture  as 
much  information  to  identify  new  data  requirements  for  standardization  (complete  with  domain 
and  range  information  for  the  allowable  data  values).  This  analysis  is  required  to  derive  the 
correct  DQE  quality  parameters,  described  below.  A  reverse  engineering  effort,  considered  an 
activity  outside  of  the  TDQM  process,  is  implemented  using  a  different  methodology. 

3.  Bsyslflg  Quality  Parameters- 


a.  If  enough  information  is  available  to  proceed,  then  the  DQE  team  should  begin 
to  conduct  an  extensive  analysis  of  the  available  information  and  develop  quality  parameters 
The  steps  are: 

(1)  Identify  and  Retrieve  Data  Element  Values  To  Be  Assessed  (Data 
Element  values,  Physical  Tables,  Specifications  For  Data 
Retrieval) 

(2)  Map  Data  Elements  To  Standard  Metadata 

(3)  Prepare  Rule  Sets  From  Business  Rules,  Systems  and  Data 
Specifications,  Policies,  Regulations  and  Procedures 
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(4)  Develop  Metrics 

(5)  Record  Analysis  Results 


b.  Identify  and  Retrieve  Data  Element  Values  to.be  Assessed,  After  the  initial 
review  of  the  system  and  data  documentation,  the  data  elements  for  the  unqualified  data  values 
are  identified,  and  a  specification  for  a  data  extract  is  prepared.  The  specification  will  include 
the  structures  and  associated  data  elements  required  to  undergo  the  DQE  process.  Selection 
parameters  will  also  be  included  if  required  (if  for  example,  it  is  required  to  subset  the  data). 
It  is  critical  that  the  download  specification  include  all  of  the  values  for  the  data  elements 
required  to  build  the  operational  rule  sets  that  will  actually  check  the  data.  In  cases  where  a 
specific  domain  and  range  cannot  be  found  in  existing  documentation,  the  DoD  DQE  team 
establishes  a  tentative  domain  and  range  that  is  based  on  a  study  of  a  series  of  actual  data  values 
and  functional  validation. 


c.  Mao  Data  Elements  to  Standard  Metadata.  DoD  DQE  specifications  are 
developed  by  following  a  structured,  carefully  designed  data  analysis  process  that  results  in  the 
establishment  of  standard  metadata  and  the  business  rule  set.  Standard,  consistent  metadata  is 
the  foundation  for  accurate  data.  Analysts  should  study  the  data  structures,  the  data  values  (the 
actual  "instances  of  data"),  and  related  models,  orders,  documents  and  regulations. 
Standardization  analysis  techniques  are  used  in  order  to  establish  standard  and  valid  definitions, 
formats,  domains,  and  ranges.  Data  modeling  techniques  are  used  to  establish  data  and  structure 
relationships  and  the  associated  business  rules.  The  following  interrelated  analysis  phases  are 
supported  throughout  by  functional  user  validation  ("Data  Quality  Engip  jerinp  (DQE)  Pilot 
Project,  Technical  Report,  reference  (p)): 


(1)  Hi£tbLgysLCQnsiSlSDgX,..AralySttt  This  series  of  analyses  identify 
inconsistencies  in  data  element  definitions  and  usage  within  a  given  database.  The  analyses 
identify  identically  named  dements  with  differing  formats,  identically  named  elements  with 
multiple  definitions,  and  differing  data  elements  having  similar  definitions. 


(2)  Collision  Analysis.  This  series  of  analyses  compares  data  element 
definitions  to  a  DoD  standard,  Joint,  functional  area,  Component,  or  other  data  elements.  The 
analyses  facilitate  later  data  exchanges  and  provide  the  foundation  for  establishing  valid 
definitions,  formats  and  domains  in  conformance  with  higher  order  standards. 
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(3)  nata  Structure  Analysis.  This  effort  establishes  relationships  between 
data  elements  based  on  prime  words,  class  words,  and  domain  and  range  specifications.  This 
study  results  in  a  logical  data  model,  provides  a  map  of  data  elements  and  constructs  across 
heterogeneous  databases,  and  establishes  the  business  rules  required  to  validate  the  data. 


(4)  Domain  and  Range  Analyses.  The  product  of  this  set  of  analyses  is 
the  set  of  legitimate  values  for  a  given  data  element.  This  is  based  on  a  study  of  actual  sets  of 
data  values  stored  in  one  or  more  databases,  the  review  of  documentation  provided  by 
authoritative  sources,  and  interviews  with  functional  and  technical  experts. 


e.  Prepare  Rule  Sets  From  Business  Rules.  Systems  and  Data  Specifications. 
Policies.  Regulations  and  Procedures.  The  standard  metadata  and  business  rules  resulting  from 
the  analysis  process  are  used  to  prepare  generic  and  specific  rule  sets.  Generic  rule  sets  are 
those  rules  (i.e.  standard  algorithms)  that  apply  to  the  data  independent  of  an  automated  system. 
The  generic  rule  is  transformed  into  specific  file-unique  rule  sets.  Specific  rule  sets  are  the 
actual  code  that  is  executed  in  the  system  or  DQE  tool  that  will  actually  perform  the  assessment. 
For  a  non-automated  assessments  the  specific  rule  may  actually  be  the  specific  operating 
procedure  that  will  be  executed  to  assess  the  quality  of  data  on  a  form  or  report.  Regardless  of 
how  the  specific  rule  is  used,  it  is  generated  from  a  generic  rule  to  quality  engineer  particular 
data.  Some  generic  rule  types  include  ("Data  Quality  Engineering  (DQE)  Pilot  Project, 
Technical  Report,  reference  (p)): 


(1)  Null  constraints  -  Testing  for  Unqualified  Zero  and  Null  Data 
Values 

(2)  Domain  and  Range  Validation  -  Testing  for  Data  Values  Outside 
Domains  and  Ranges 

(3)  Operational  Rule  Sets  -  Testing  for  Data  Values  Not  Complying 
With  Calculated  Values 

(4)  Relationship  Validation  -  Testing  for  Data  Values  Not  Compliant 
With  Business  Rules 

(5)  Statistical  rests  -  Testing  for  Data  Values  +/-  2  Standard 
Deviations  From  a  Calculated  Data  Element  Norm  (Numerical 
Fields  Only) 
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f.  DQE  generic  and  specific  rule  sets  should  be  generated  for  each  data  element 
to  be  checked.  For  an  example  cf  ruie  set  generation,  see  Table  4-2.  The  development  of 
generic  and  specific  ruie  sets  is  an  iterative  process  that  depends  on  functional  and  technical 
expert  validation.  Many  times  the  validation  and  refinement  of  specific  rules  will  occur  after  the 
data  values  are  measured.  The  specification  indicates  the  valid  format  for  the  data  element,  the 
valid  domain  and  range,  the  horizontal  relationships  that  the  element  participates  in  across  the 
record  (the  application  of  business  rules  to  the  element  set),  any  cross  record  relationships 
(referential  integrity  and  relationship  checks),  and  any  vertical  relationships  that  the  element 
participates  in  within  its  particular  column  (the  application  of  statistical  and  anomaly  testing) - 
Generic  DQE  rules  should  be  recorded  in  the  DDRS  or  dictionary  and,  in  the  next  step, 
implemented  as  specific  rules  sets.  There  is  no  limit  on  the  number  of  specific  rules  that  can  be 
defined  to  check  the  data  values  in  a  given  field. 


Historical  Data 
Problem  Statement 

Rule  Type 

Generic  Ruie  Set 

The  equipment 
identifier  fields  are 
often  blank. 

Nail 

Constraints 

If  the  equipment  identifier 
is  zero,  blank,  or  null  then 
error. 

Specific  Rule  Set 

Select  equip_id  where 
equip Jd  ■»  0  or 
equip_id  »'  '  or 
equipjd  is  NULL; 


Table  4-2  DoD  DQE  Rule  Set  Generation 


g.  Develop  Metrics.  Metrics  (specific  measures  that  indicate  how  "good"  the  data 
are)  are  developed  as  an  integral  component  of  the  data  analysis  phase.  As  with  the  rule  sets, 
there  is  a  set  cf  metric  types  that  are  defined  genetically.  As  a  particular  file  is  analyzed,  the 
generic  metric  types  are  tailored  to  work  with  that  file.  The  generic  metric  types  are  highly 
analogous  to  the  rule  types  and  include  the  following  ("Data  Quality  Engineering  (DQE)  Pilot 
Project.  Technical  Report,  reference  (p)): 


(1)  Consistency  Measures 

(2)  Redundancy  Measures 

(3)  Completeness  Measures 
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(4)  Valid  Value  Conformance  Measures 

(5)  Relationship  Rule  Conformance  Mervures 

(6)  Referential  Integrity  Conformance  Measures 


h.  During  the  MEASURE  activity,  the  data  are  assessed  against  a  goal  for  the 
metric.  For  example,  in  most  cases  it  is  desirable  for  the  referential  integrity,  consistency, 
completeness,  valid  values,  and  relationship  rules  measures  to  strive  toward  a  goal  of  100 ft 
conformance  with  quality  specifications,  while  redundancy  measures  should  target  a  Oft 
assessment.  These  metrics  are  used  to  establish  the  data  quality  baseline.  The  BQE  team 
should  determine  the  display  tools  that  will  be  used  for  reporting  and  analyzing  the  data  quality 
baseline.  Such  tools  may  include,  Pareto  Chans,  Histograms,  Pie  Charts,  Tables,  and  Control 
Cham.  Refer  to  Chapter  5  of  this  Manual  for  more  information  on  these  tools  and  techniques. 


i.  Record  Analysis  Results.  The  structured  analyses  should  provide  the  DQE  team 
with  an  expert  knowledge  of  the  data  baseline.  Throughout  tht  DEFINE  activity,  the  DoD  DQE 
team  should  be  capturing  and  recording  standard  or  standards  compliant  metadata.  The  metadata 
includes  data  element  definitions,  formats,  general  and  specific  domains  and  ranges,  and  the 
logical  data  model  (including  the  business  rule  set).  Wherever  possible,  new  data  requirements 
should  be  forwarded  for  standardization  in  accordance  with  DoD  8320.1 -M-l  (reference  (k)). 


D.  MEASURE 


1 ,  The  purpose  of  the  MEASURE  activity  is  to  determine  the  exact  nature  and  magnitude 
of  problems  with  ths  real  data  values.  The  objectives  of  the  steps  are  to  prepare  tools  and 
procedures  for  performing  the  measures,  assess  the  data  values  against  the  quality  parameters, 
record  and  validate  suspect  data  errors,  refine  the  rule  sets  and  calculate  the  data  quality  metrics. 
It  is  at  this  phase  that  the  DQE  team  establishes  the  data  quality  baseline  assessment.  Functional 
and  user  feedback  is  vital  at  this  stage,  to  ensure  the  assessment  data  is  valid  and  depicts  ran 
accurate  picture  of  the  real  data  quality  problems.  The  steps,  tods,  and  techniques  of  the 
MEASURE  activity  are  presented  in  Table  4-3  below. 
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OBJECTIVES 

MEASURE  ACTIVITY  STEPS 

TOOLS/ 

TECHNIQUES 

Measure  and 

1.  Configure  and  Run  DQE  Tool  (If 

DDRS 

assess  the  data 

Available) 

Data  Model 

quality. 

Benchmarking 

2.  Apply  Rule  Sets  To  Data  Instances 

Control  Chart 
DTR 

3.  Flag  Suspect  Data 

Pareto  Chart 
Histogram 

5.  Review  Rule  Set  Validity,  If  Not  Valid, 

DQE  tool 

Redefine 

Checksheets 

4.  Calculate  Metrics  To  Produce  Reports 

Table  4-  3  DoD  DQE  MEASURE  Activity 


2.  Configure  and  Run  DQE  Tool.  DQE  tools  are  automated  or  non -automated 
applications  that  aid  in  the  prevention,  detection,  correction,  and  monitoring  of  data  quality 
problems.  In  order  to  assure  data  quality,  the  most  cost  effective  activity  is  to  prevent  initial 
defects.  Therefore,  technical  developer;  should  build  data  quality  tools  into  the  design  of  a 
database  and  all  of  its  interfaces.  Currently  some  Database  Management  System*  (DBMS)  or 
application  programs  already  provide  Internal  tools  to  aid  in  maintaining  quality  data.  In  most 
cases,  this  consists  of  data  quality  checks  being  performed  as  edits  on  data  entry.  Since  DBMS 
and  application  programs  are  not  designed  to  perform  data  quality  checks  whenever  the  data  are 
manipulated,  copied,  transferred,  or  converted,  separate  analysis  and  correction  (i.e.  DQE)  tools 
are  needed  by  technical  experts  to  detect  and  correct  errors  after  they  have  been  stored  in  the 
system.  The  advantage  of  an  automated  DQE  tool  is  that,  it  is  a  data-driven  architecture  that 
can  be  reconfigured  quickly  and  easily  to  input  new  file  specifications  and  DQE  specific  rule 
sets.  But,  as  DoD  legacy  systems  migrate  to  target  systems,  technical  developers  must  build 
data  quality  engineering  into  the  design  of  the  new  system,  thereby  phasing  out  the  need  for 
separate  DQE  tools.  For  non-automated  data  quality  engineering  efforts,  the  DQE  team  should 
establish  standard  operating  procedures  with  users  for  detecting,  documenting,  and  reporting 
errors.  The  team  should  establish  the  measurement  technique  (i.e.  sampling)  and  design  a  Daw 
Trouble  Report  (DTR)  along  with  instructions  for  use.  The  automated  DQE  tool  should  be 
configured  to  produce  graphic  and  tabular  data  error  reports. 
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3.  fluk  Oft  Tn  Data  Instances.  Once  configured,  the  DQE  automated  tool  or 

non-automated  procedures  should  be  implemented  to  process  the  specific  rule  sets  against  real 
data  instances.  Data  values  are  subject  to  checks  based  on  the  operationally  onented 
(if... then...)  rule  sets,  identified  in  the  DEFINE  activity.  The  rules  sets  are  executed  to  check 
for  invalid  null  constraints,  duplicate  records,  and  invalid  domains  and  ranges.  The  rule  sets 
are  also  used  to  check  for  errors  by  comparing  a  given  data  value  to  a  calculated  value  using 
related  data  in  a  given  record  or  database.  For  non-automated  procedures,  the  rule  sets  should 
be  applied  to  data  values  from  existing  hardcopy  files;  or  the  rules  may  have  to  be  applied  over 
a  time  period  to  new  data  as  it  is  being  collected  in  the  functional  process. 


4.  Flap  Suspect  Data.  As  the  rule  sets  are  processed,  the  suspect  data  values  should  be 
extracted  from  the  data  file  along  with  information  on  the  data  elements  that  fail  one  or  more 
DQE  checks.  All  of  the  flagged  suspect  data  should  be  compiled  to  create  a  Data  Error  Report. 
The  Data  Error  Reports  should  be  documented  in  the  appropriate  section  of  the  Data  Quality 
Baseline  Assessment  Report.  For  non-automated  checking,  as  suspect  data  is  identified,  each 
error  should  also  be  documented  on  a  Data  Trouble  Report  (DTR).  An  automated  tool  could 
be  used  at  this  point  to  create  and  submit  the  DTRs  to  the  DQE  leader.  Both  the  data  error 
report  and  the  DTRs  are  used  in  the  ANALYZE  and  IMPROVE  activities  to  track  and  monitor 
corrective  actions. 


5.  Review  Rule  Set  Validity.  If  Not  Valid.  Redefine.  The  MEASURE  activity  is  an 
iterative  process.  The  data  analyst  should  work  with  the  data  experts  to  review  and  validate  the 
flagged  errors  as  true  daa  errors.  In  many  cases  the  rule  set  is  just  too  generic  causing  it  to 
need  adjustments  in  order  to  capture  valid  errors.  As  a  result,  the  generic  and/or  specific  rule 
set  should  be  refined  and  the  tool  reconfigured  with  minimal  effort.  But,  there  may  be  times 
when  more  information  is  required  from  the  DEFINE  activity  because  initial  historical  problems 
were  not  sated  properly  or  they  were  inaccurately  transformed  into  quality  parameters.  This 
results  in  the  dau  analyst  going  back  to  the  DEFINE  activity  to  further  the  analysis  and 
redesigning  of  the  quality  parameters. 


6.  Calculate  Metrics  To  Produce  Reports.  Once  the  measurement  is  complete,  all  of  the 
flagged  suspect  daa  should  be  grouped  into  failure  categories,  tallied,  and  used  to  calculate  the 
metrics.  The  results  from  the  metrics  calculation  depict  the  overall  results  of  the  MEASURE 
activity  and  are  considered  the  baseline  assessment  daa.  The  baseline  assessment  dau  displays, 
in  quantifiable  terms,  how  good  or  bad  the  daa  quality  really  is.  The  automated  DQE  tool 
provides  the  capability  to  perform  identically  configured  checks  in  specific  time  intervals.  This 
provides  an  ideal  method  of  producing  graphical  reports  based  on  cunent  and  previously 
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established  baseline  assessment  data.  These  reports  can  be  used  to  measure  progress  made 
towards  achieving  data  quality  ever  time.  These  reports  should  be  formally  documented  in  the 
appropriate  section  of  the  Data  Quality  Baseline  Assessment  Report. 


£.  analyze 


1.  The  purpose  of  the  ANALYZE  activity  is  to  identify  and  analyze  root  causes  for  the 
occurrences  of  data  errors  and  to  identify  opportunities  for  improvement  One  objective  of  the 
steps  of  this  activity  is  to  employ  the  assistance  of  functional  and  technical  data  experts  who  are 
most  familiar  with  the  datt  systems  and  processes,  to  identify  and  validate  possible  root  causes. 
This  activity  also  includes  analyzing  the  impact  of  poor-quality  costs  to  identify  solutions  for 
dan  eiTor  problems  and  providing  recommendations  for  improvement  opportunities.  The  steps, 
tools,  and  techniques  of  the  ANALYZE  activity  are  p wwnwi  in  Table  4-4  below. 


I  OBJECTIVES 

ANALYZE  ACTIVITY  STEPS 

TOOLS/ 

TECHNIQUES 

I  Identify  root 

1.  Identify  Sources  Of  Erroneous  Dan 

DDRS 

I  causes  of  dan 

Data  Model 

errors  and 

2.  Contact  and  Involve  Functional  Dan 

Activity  Model 

analyze 

Steward,  Along  With  Other 

NGT 

opportunities 

Functional,  Technical,  and  Dan 

Cost  of  Quality 

for 

Expens 

Control  Chan 

improvement. 

DTR 

3.  Analyze  Dau  Value  Problems/Errors 

Pareto  Chart 

To  Identify  Root  Cause  Of  Error 

Histogram 

DQE  Tool 

4.  Develop  Poor-Quality  Cost  Baseline 

Chccksheets 

For  Root  Causes 

S.  Determine  Recommended  Solution 
Alternatives  and  Improvements  for 

Cause  Jc  Effect 
Process  Flow 

1. 

Root  Causes 

Table  4-4  DoD  DQE  ANALYZE  Activity 
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2.  Tdgnrifv  Sources  Of  Emma ous  Data.  The  first  step  in  implementing  an  effective 
corrective  action  is  to  identify  the  data  source,  and  the  data  steward.  The  DoD  DQE  team 
examines  suspect  data  where  more  than  one  source  for  similar  data  exists  and  if  necessary, 
determines  the  best  source  for  the  data  based  on  the  quality  assessments  provided  by  the  DoD 
DQE  tool.  Data  stewards  or  expens  are  determined  to  identify  the  correct  people  required  to 
support  the  DQE  team  in  resolving  dan  problems. 


3.  Contact  and  Involve  Functional  Data  Steward.  Alone  With  Other  Functional. 
TVrhmrai  and  Data  Experts.  Individual  efforts  can  substantially  contribute  to  the  data  quality 
assurance  effort,  even  if  undertaken  outside  the  context  of  the  DQE  team.  It  is  important  to 
contact  and  involve  the  functional  data  stewards  and  functional,  technical,  and  data  expens  who 
work  with  the  data  or  axe  in  the  process  every  day.  The  DQE  team  must  depend  on  them  to 
identify  causes  of  problems  with  their  daa  and  solutions  or  opportunities  for  improvements. 
People  will  contribute  most  when  they  axe  responsible  for  something.  Therefore,  if  these 
individuals  recognize  that  quality  is  their  responsibility  every  day,  it  will  become  easy  to  accept 
thr  importance  of  being  involved  and  providing  resources  necessary  to  improve  data  quality. 
The  DQE  team  has  a  responsibility  to  encourage  each  individual  to  contribute,  recognize  each 
contribution,  no  matter  how  small,  and  consult  with  every  individual  as  appropriate.  The  goal 
is  to  develop  a  climate  for  each  DQE  effort  in  which  people  are  increasingly  willing  to 
participate  actively  in  the  improvement  activity. 


4.  Analyze  Daa  Value  Problcms/Errors  To  Identify  Root  Cause  Of  Error.  For  each  daa 
value  error  category  generated  by  the  DQE  tool,  analysts  determine  a  likely  or  probable  root 
cause.  The  root  cause  analysis  process  begins  with  an  assessment  of  the  dau  in  a  very  broad 
aspect.  Remember  to  seek  out  true  causes  of  problems  and  not  on  the  symptoms.  Some  key 
questions  to  answer  are:  In  what  areas  did  a  significant  number  of  errors  occur?  Did  a  certain 
type  of  error  occur  with  some  frequency?  What  is  the  best  area  on  which  to  concentrate  efforts 
so  as  to  get  the  biggest  improvement  in  data  quality?  Analysis  of  errors  that  occurred  on  a  more 
scattered  basis  will  show  the  cause  of  the  specific  error,  but  may  not  identify  a  broad-based 
systematic  problem.  However,  scattered  errors  should  not  be  simply  fixed  on  an  individual 
bests  and  be  forgotten.  An  across-the-board  analysis  of  these  errors  may  also  be  .  vealing. 
Based  on  this  analysis,  possible  causes  of  the  data  value  problem  can  be  determined.  The  DQE 
team  should  examine  the  possible  causes  with  several  points  of  view.  In  determining  the  root 
cause,  the  DQE  team  should  test  and  validate  the  possible  causes.  The  Cause  and  Effect  and 
Process  Flow  techniques  can  be  used  for  identifying  root  causes, 
classified  into  one  or  more  of  the  four  major  problem  categories: 

Problem,  Policy  and  Procedure  Problem  or  Data  Design  Problem  ("Data  Quality  Engineering 
(DQE)  Pilot  Project,  Technical  Report,  reference  (p)). 


Possible  causes  should  be 
Process  Problem,  System 
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a.  Process  Problem  •  Past  experience  has  revealed  that  the  majority  of  data  error 
problems  can  be  attributed  to  a  process  problem.  For  data  errors  categorized  as  process 
problems,  effort  is  placed  on  analysis  of  existing  process  models,  and  using  that  knowledge  base 
to  find  and  recommend  actions  to  correct  the  process  fault. 


b.  System  Problem  -  Data  problems  often  stem  from  system  design  and  use, 
particularly  in  the  case  of  'legacy*  systems.  In  the  typical  operation,  legacy  systems  art 
meeting  functional  requirements  in  spite  of  numerous,  poorly  doosmented  modifications.  On 
careful  examination,  corrupt  data  values,  data  sources  that  art  not  clearly  defined,  the  system 
being  used  to  support  requirements  beyond  the  original  intent,  and  a  systems  architecture  that 
defines  a  system  integration  effort  can  usually  be  found. 


c.  Policy  and  Pmcartnrr  Problem  •  An  analysis  of  data  value  errors  often  reveals 
either  conflicting  guidance  in  current  policy  and  procedure,  twnr  of  appropriate  guidance,  or  a 
failure  to  comply  with  published  policy/procedure. 


d-  Data  Design  Problem  -  There  is  also  the  potential  that  data  models  or 
standardized  metadata  will  either  need  to  be  modified  in  order  to  correct  what  appear  to  be 
persistent  relationship  problems,  or  new  data  requirements  actually  developed  in  order  to 
formally  specify,  document  and  standardize  the  appropriate  data  structures  and  uses. 


e.  Some  data  value  errors  cannot  be  attributed  to  a  specific  faulty  process,  system 
error  or  policy/procedure.  The  remedial  action  in  such  cases  is  to  analyze  the  data  error  and 
determine  the  corrective  action. 


5-  BgyglOP  Poor-Quality  Cost  fPQO  Baseline  For  Root  Causes. 


a.  A  Poor-Quality  Cost  (PQC)  baseline  must  be  established  for  the  root  causes  in 
order  to  identify  opportunities  for  improvement  or  to  assess  data  quality  improvements.  PQC 
is  defined  as  all  of  the  costs  incurred  to  create  and  maintain  the  data  right  every  time  and  the 
cost  of  determining  if  the  data  values  are  acceptable,  plus  any  cost  incurred  by  the  organization 
and  the  end-user  because  the  data  did  not  meet  requirements  and/or  end-user  expectations.  The 
term  PQC  is  appropriate  because  the  baseline  is  designed  to  help  identify  and  reduce  the  cost 
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associated  with  poor  quality.  Poor  quality  costs  the  DoD  money.  Good  quality  will  save  the 
DoD  money.  Therefore,  cost  baselines  are  not  required  for  good  quality  daa.  The  importance 
of  PQC  lias  already  been  recognized  by  DoD  when  a  requirement  for  PQC  systems  was  included 
in  Military  Standard  MH.-Q-9858A  ("Poor-Quality  Cost",  reference  (s)). 


b.  In  general,  PQCs  are  categorized  into  four  areas:  prevention,  appraisal,  internal 
failure,  and  external  Mure.  Once  these  costs  are  identified,  there  can  be  a  better  understanding 
of  what  the  costs  are,  and  funds  can  be  targeted  to  the  areas  with  the  greatest  potential  for 
improvement  and  cost  savings.  The  following  sections  describe  procedures  on  how  to  establish 
the  PQC  baseline. 


c.  Determine  Activity  Costs.  The  PQC  baseline  provides  a  very  useful  tool  to 
change  the  way  both  management  and  employees  think  about  errors.  In  most  companies,  the 
activities  that  foster  the  costs  of  administrative  errors  and  the  resulting  checks  and  balances  are 
accepted  as  a  way  of  life.  Applying  PQCs  to  these  activities  focuses  management’s  attention  on 
the  activities  that  cause  this  neglected  waste.  An  activity  is  a  named  process,  function,  or  task 
that  occurs  over  time  and  nas  recognizable  results.  Activities  are  considered  the  building  blocks 
for  assuring  data  quality.  Therefore,  it  is  essential  to  understand  these  activities  in  order  to 
implement  data  quality  improvement.  The  technique  used  in  DoD  for  looking  at  activities  and 
understanding  them  is  called  Activity  Based  Costing  (ABC)  ("Corporate  Information 
Management,  Process  Improvement  Methodology  For  DoD  Functional  Managers",  reference 
(0).  ABC  shows  managers  what  they  do  with  their  money.  Through  the  use  of  ABC,  the  DQE 
team  must  define  the  activities  for  data  quality  and  determine  the  costs  associated  with  these 
acuvities. 


(1)  Analyze  Activities.  ABC  begins  with  identifying  the  activities  that 
apply  to  managing  the  data.  The  scope  of  the  activities  to  be  analyzed  should  be  based  on  the 
specific  data  elements  where  data  errors  occurred  in  the  MEASURE  activity.  The  identification 
of  the  activities  is  the  most  difficult  tasks  within  ABC.  The  IDEFO  activity  modeling  technique 
provides  a  structured  approach  to  the  identification  and  analysis  of  the  activities.  The  DQE  team 
can  use  the  activity  model  as  a  basis  for  interviewing  key  people  associated  with  the  business 
process  and  researching  the  accounting  data,  such  as  finding  out  the  percentage  of  time  involved 
in  each  activity.  Through  ABC,  functional  managers  can  characterize  the  value  of,  or  need  for, 
each  activity.  In  turn,  these  characterizations  can  be  used  to  rank  activities  for  improvements. 
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(2)  Determine  Costs  of  Activities.  Secondly,  ABC  identifies  the  costs  that 
are  associated  with  the  different  activities.  This  ability  to  match  costs  with  activities  quickly 
highlights  where  improvement  is  reeded,  whether  one  is  determining  long-term  data  quality 
improvement  priorities,  or  measuring  short-term  success.  Gathering  the  cost  data  can  begin 
concurrently  with  the  step,  "analyze  activities."  Typically,  historical  costs  are  used  for  the 
initial  baseline.  Once  the  baseline  is  established,  the  selected  improvement  opportunity  should 
include  a  plan  for  collecting  current  costs  to  update  the  PQC  baseline.  Costs  are  normally 
obtained  from  numerous  existing  records  (e.g.  financial  reporting  system,  salary  information, 
budgets,  invoices).  The  costs  may  have  to  be  estimated  or  calculated  based  on  salary  and  level 
of  effort  information.  The  thrust  of  PQC  should  be  to  accumulate  the  total  PQC  figures.  This 
allows  for  general  assumptions  that  may  have  to  be  refined  later  during  the  improvement 
process.  The  PQC  baseline  is  a  management  tool  for  improvement,  and  its  use  does  not  require 
precise  costs. 


d.  Establish  the  PQC  Baseline.  The  main  types  of  PQC  costs,  depicted  in  Table 
4-5  below,  are  direct  PQCs  and  indirect  PQCs  ("Poor-Quality  Cost",  reference  (s)).  Direct 
PQCs  include  the  costs  the  organization  incurs  because  management  is  afraid  that  people  will 
make  mistakes,  all  of  the  costs  incurred  because  data  errors  are  detected,  and  the  costs  related 
to  training  people  and  maintaining  systems  so  the  data  is  managed  effectively.  Indirect  PQCs 
are  defined  as  those  costs  not  directly  measurable  in  the  financial  reports,  but  are  part  of  PQC. 
They  consist  of  three  major  categories:  Customer-incurTed  PQC,  Customer-dissatisfaction  PQC, 
and  Loss-of-reputation  PQC.  The  PQC  baseline  should  primarily  focus  on  the  direct  PQCs. 
Direct  PQCs  are  better  understood,  are  traditionally  used  by  management  to  run  the  business, 
and  the  results  are  less  subjective.  Also,  the  direct  costs  are  usually  found  in  financial  reports 
where  they  can  be  verified.  However  the  indirect  PQCs  should  be  identified  wherever  possible. 
DoD  organizations  must  consider  the  impact  of  poor  quality  data  on  their  customer  (e.g.  the 
Warfighter). 
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DIRECT  POOR-QUALITY  COSTS 

INDIRECT  POOR-QUALITY  COSTS 

A.  Controllable  poor-quality  cost 

A.  Customer-incurred  cost 

1.  Prevention  cost 

I  2.  Appraisal  cost 

B.  Resultant  poor-quality  cost 

B.  Customer-dissatisfaction 

1.  Internal  error  cost 

cost 

2.  External  error  cost 

C.  Equipment  poor  quality  cost 

C.  Loss-of-reputation  cost 

Table  4-5  Type  of  Poor-Quality  Cost 


(1)  Classify  Activities  into  Direct  PQCs.  Once  the  activity  based  costs  are 
established,  differentiate  the  activity  costs  between  operational  (business)  costs  and  quality 
activity  costs  (i.e.  differentiate  the  activities  that  are  done  to  ensure  the  quality  of  the  products 
from  the  operational  activities).  Quality  activities  should  be  classified  into  the  direct  PQC 
categories.  Direct  PQCs  encompass  three  major  types  of  expenditures:  Controllable  PQCs, 
Resultant  PQCs  and  Equipment  PQCs.  Controllable  PQCs  are  those  that  management  have 
direct  control  over  to  ensure  that  only  end-user  acceptable  data  are  delivered  to  the  end-user. 
Controllable  costs  a tt.  considered  prevention  and  appraisal  cost.  Resultant  PQCs  (i.e.  losses) 
include  all  incurred  costs  that  result  from  the  errors  and  are  directly  related  to  the  controllable 
PQC  category.  This  includes  the  money  spent  because  the  data  was  not  created  or  maintained 
right.  The  resultant  costs  are  considered  internal  failure  costs  and  external  failure  costs. 
Equipment  PQCs  include  the  investment  in  the  DQE  tools  used  to  define,  measure,  analyze,  or 
improve  the  data,  plus  the  cost  of  the  space  that  equipment  occupies.  This  includes  the  cost  of 
the  equipment  used  to  print  and  report  quality  assessment  data.  Table  4-6  depicts  example 
activity  costs  components  for  these  areas  ("Pool -Quality  Cost",  reference  (s)).  There  are  times 
where  one  activity  may  span  more  than  one  area.  In  this  case,  indicate  a  percentage  of  the 
activities  cost  in  the  appropriate  cost  category. 


(3)  Calculate  PQC  Baseline  Data.  Once  the  activities  have  been 
categorized  into  one  or  more  of  the  direct  PQC  areas,  then  cost  data  in  each  category  should  be 
tallied,  and  used  to  calculate  the  cost  baseline.  Calculating  the  cost  baseline,  entails 
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understanding  how  the  PQC  elements  interact  with  one  another.  The  overall  objectives  for  any 
data  quality  assurance  program  are  to  reduce  the  number  of  data  errors,  reduce  the  resultant 
costs  (i.e.  internal  and  external  failure  costs)  and  to  reduce  the  overall  costs  of  quality  activities. 
But,  in  order  to  obtain  these  objectives,  the  controllable  costs  (prevention  and  appraisal  costs) 
will  need  to  fluctuate.  The  quality  baseline  can  be  depicted  from  various  angles.  Figure  4-2 
illustrates  the  interaction  between  the  cost  elements  versus  time  as  one  example  of  this  ('Poor- 
Quality  Cost",  reference  (s)).  When  the  organization  begins  performing  appraisals,  the  appraisal 
cost  will  rise  due  to  setting  up  the  measurement  system.  As  the  appraisals  are  conducted  the 
external  failure  costs  will  begin  to  decline  because  less  errors  will  be  delivered  to  the  end-user; 
but,  the  internal  failure  costs  will  begin  to  fluctuate  up  and  down  as  part  of  the  continuous 
improvement  cycle  (i.e.  when  new  errors  are  detected  and  corrective  actions  are  taken  to  resolve 
the  root  causes).  As  improvement  projects  are  implemented,  the  prevention  costs  should  begin 
to  rise;  but,  as  the  data  quality  is  improved  the  appraisal  costs  and  internal  costs  will  begin  to 
decline  because  the  need  to  measure  will  be  replaced  by  improved  systems,  processes,  policies, 
and  data  design.  Overall,  the  total  cost  of  quality  begin  to  decline.  PQC  will  never  drop  to  zero 
because  some  prevention  activities  should  always  be  performed  and  some  appraisal  activities  will 
be  needed  to  provide  management  with  data  quality  assurances.  The  PQC  baseline  can  also  be 
represented  as  cost  versus  the  number  of  errors.  Figure  4-3  depicts  the  effect  of  the  prevention 
cost  on  the  total  number  of  errors  and  the  total  error  cost  ("Poor-Quality  Cost",  reference  (s)). 
Finally,  the  baseline  data  should  be  reported  in  a  tabular  or  graphical  representation  in  the  Data 
Quality  Baseline  Assessment  Report.  (See  Appendix  B  for  the  content  outline  of  a  Data  Quality 
Baseline  Assessment  Report.) 


e.  This  cost  baseline  is  not  intended  to  be  completely  accurate,  but  rather  an 
estimate  of  costs  to  indicate  and  understand  the  costs  of  poor  quality.  The  cost  baseline  is  not 
a  financial  report  that  will  be  audited.  It  is  a  trend  and  magnitude  indicator.  It’s  purpose  is  to 
define  major  problem  areas  and  to  get  management's  attention.  Therefore  it  is  not  intended  to 
be  1007c  accurate;  in  fact,  a  good  way  to  kill  a  quality  program  is  to  insist  on  an  absolutely 
accurate  baseline.  A  cost  baseline  that  is  80%  accurate  may  serve  the  purpose  as  long  as  it  is 
constant  and  covers  the  major  activities.  With  PQC  reporting,  consistency  is  the  most  important 
factor. 
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CONTROLLABLE  COSTS 

RESULTANT  COSTS 

1  PREVENTION 

APPRAISAL 

INTERNAL 

EXTERNAL 

1  -  - 

FAILURE 

FAILURE 

Training  Costs 

Quality  Audits 

Scrap  and  Rework 

Complaints, 

Rebates,  Damage 

Quality  Planning 

Monitoring 

Overtime,  Idle 

Claims 

Improvements 

Time 

Quality  Assurance 

Lost  Management 

Data  Entry 

Testing  Hardware 

Malfunction  and 

Time 

and  Software 

Rerun  Costs, 

Process 

Domino  Effect 

Lost  Business, 

Improvements 

Inspection  of 

From  Failure 

Customer  Affairs, 

Purchased  Items 

Customer  Goodwill 

Procedure 

Preparation 

Quality  Analysis 

Retesting 

Failure  Analysis 

Cost  to  Customer 
Due  to  the  Failure 

System 

Quality  Assurance 

Maintenance 

Data  Processing 

Corrective  Actions 

Costs  Indirectly 

Configuration 

to: 

Impacted  From 

Installation 

Failure 

Management 

Building 

Testing 

Quality  Data 

-  Data  Values 

-  Process 
*  System 

Lost  Opportunity 
Unrealized  Savings 

Generalized 

Collection  and 

-  Documentation 

Application 

Analysis  Operations 

Incorrect  Decisions 

Modules,  Reusable 

Based  on  Wrong 

Code 

System  Assurance 
Reviews 

Productivity  Loss 

Information 

Preparation  for 

Security  Exposure 
Losses 

Testing 

Legal  Exposure 
Losses 

Lost  Assets 

Table  4-6  Examples  of  Indirect  PQCs 
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Figure  4-3  Effect  of  Prevention  Costs  on  the  Total  Number  of  Errors 
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6.  Determine  Re^mmenderi  Solution  Alternatives  and  Improvements  fgl  RPOt  CaUSCS- 


a.  This  is  where  the  objective  of  the  ANALYZE  step  is  actually  met  and  the  drive 
for  improvement  begins.  The  prior  steps,  including  measuring,  identifying  root  cause,  and 
esablishing  a  PQC  baseline  should  have  helped  the  team  to  get  an  overall  picture  of  the  data 
quality  problems.  The  results  should  not  be  looked  at  negatively  or  as  indications  of  failure. 
Problems  provide  a  chance  for  improvement  and  serve  as  gold  mines  for  potential  savings. 
Acknowledging  problems  and  rewarding  those  who  bring  problems  to  light  is  an  essential  part 
of  the  data  quality  assurance  process. 


b.  The  DQE  team  should  look  at  emergent  or  long-term  problems  for  potential 
improvement  projects.  For  each  root  cause,  the  DQE  team  will  need  to  analyze  the  data  quality 
baseline  data,  the  root  causes  and  the  PQC  baseline  to  determine  recommended  solutions. 
Innovative  thinking  should  be  encouraged  in  order  to  come  up  with  cost  effective  new  ideas. 
The  end-user  should  be  consulted  for  often  times  they  have  already  considered  or  suggested 
possible  solutions.  Small  incremental  improvement  ideas  are  favored  for  they  am  usually  quick 
and  easy  to  implement.  If  several  recommended  alternatives  are  available,  use  the  quality  and 
cost  baselines  to  justify  the  feasibility  of  each  recommendation.  Data  quality  baselines  are  useful 
for  identifying  solutions  whereby  the  least  amount  of  effort  can  be  expended  to  reduce  the  most 
number  of  errors.  The  PQC  baseline  is  useful  for  determining  the  least  amount  of  money  that 
can  be  expended  to  reduce  the  most  number  of  errors.  The  cost  baseline  is  also  a  good  tool  for 
justifying  the  need  to  spend  more  money  on  controllable  activities  (i.e.  prevention  and  appraisal 
costs).  Sometimes,  the  only  solution  is  to  identify  new  or  modified  data  requirements.  If  so, 
the  metadata  solution  will  have  to  comply  with  DoD  8320. 1-M-l  (reference  (k))  rules  for  data 
element  standardization. 


F.  IMPROVE 


1.  The  purpose  of  the  IMPROVE  activity  is  to  implement  improvement  initiatives  for 
correcting  and  further  preventing  data  defects.  The  objectives  of  the  steps  are  to  select  the 
recommended  solutions  and  improvement  alternatives,  plan  for  improvement,  implement 
solutions,  check  for  improvement,  and  act  to  institutionalize  the  improvements.  The  steps,  tools, 
and  techniques  of  the  IMPROVE  activity  are  presented  in  Table  4-7  below. 
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IMPROVE  ACTIVITY  STEPS 

TOOLS/ 

TECHNIQUES 

1 

Implement 

1.  Formally  Provide  Recommended 

Cost  of  Quality 

improvement 

Improvement  Alternatives  and 

QFD 

initiatives  for 

Corrective  Actions 

DTR 

correcting  and 

Pareto  Chart 

further  preventing 

2.  Select  Improvement  Opportunity 

Histogram 

data  defects. 

and  Decide  on  Corrective  Action 

DQE  Tool 

Cause  &  Effect 

3.  Implement  Selected  Improvement 
Opportunity,  Take  Corrective 
Action  anchor  Respond  To 
Recommendations 

4.  Monitor  Implementation  and 

Process  Flow 

Corrective  Action  Progress 

5.  Validate  and  Document 

Improved  Data  Quality 

Tabic  4-7  DoD  DQE  IMPROVE  Activity 


1.  FormalW-Providc  Recommenced  Improvement  Alternatives  and  Corrective  Actions. 
In  order  to  provide  recommended  improvement  alternatives,  the  DQE  team  must  set  the  stage 
for  improvement.  Fortunately,  the  individuals  that  are  ultimately  responsible  for  implementing 
the  recommendations  were  involved  in  the  ANALYZE  activity  and  participated  in  preparing  the 
recommendations.  Setting  the  stage  requires  the  support  of  these  same  individuals  and  their 
management.  The  DQE  team  must  formally  provide  recommended  improvement  alternatives 
to  one  or  all  of  the  following  (when  applicable):  data  stewards,  Functional  Activity  Program 
Managers  (FAPM),  Automated  Information  Systems  Program  Managers  (AIS  PM),  and  users 
depending  on  the  type  of  problem  (i.c.  system,  process,  procedure  or  data  design).  Regardless 
of  the  type  of  problem,  the  data  value  error  should  be  formz'ly  documented  and  provided  to  the 
source  file  data  steward  or  DBAd  for  immediate  corrective  action.  The  documentation  should 
identify:  where  the  error  occurred  (e.g.  file,  record,  field,  name),  why  it  occurred  (compare 
with  business  rules),  and  the  probable  correction  with  cost/impact  statement. 
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2.  Select  Improvement  Opportunity  and  Decide  on  Corrective  Action,  The  individual 
receiving  the  recommendation  should  select  an  improvement  opportunity.  This  involves 
reviewing  the  potential  opportunities,  prioritizing  them,  and  choosing  the  solution  that  will 
impact  the  majority  or  all  of  the  root  causes  and  has  the  greatest  potential  for  success.  Other 
things  to  consider  include  how  the  solutions  will  affect  end-users  and  the  people  actually 
involved  in  the  process,  the  resources  needed  to  accomplish  the  improvement,  and  the  impact 
on  the  PQCs  if  the  alternative  is  not  selected.  In  addition  to  selecting  an  improvement 
opportunity  to  prevent  further  errors,  a  decision  must  be  made  the  immediate  corrective 
action  to  be  applied  to  the  current  data  defect. 


3.  Implement  Selected  Improvement  Opportunity.  Take  Corrective  Action  and/or  Respond 
To  Recommendations.  If  an  improvement  opportunity  is  selected,  the  individuals  responsible 
for  implementing  the  solution  should  quickly  move  to  implement  and  properly  document  the 
solution.  During  implementation,  it  is  important  to  ensure  that  applicable  documentation  are 
modified  appropriately.  Also,  it  is  important  to  establish  a  process  for  detecting  failures  and 
monitoring  improvements  to  ensure  that  the  changes  to  the  system,  process,  procedure,  or  data 
design  have  tneir  desired  lasting  effect  in  accordance  with  the  new  standard.  During  this  step, 
the  current  data  defect  is  also  corrected  The  data  steward  or  DBAd  should  notify  the  DQE 
team  when  the  error  is  corrected.  If  a  solution  alternative  or  corrective  action  cannot  be 
implemented,  then  an  explanation  and  solution  alternatives  should  be  provided  to  the  DQE  team. 
The  DQE  team  should  document  all  implemented  solutions,  corrective  actions  taken,  and 
explanations  in  the  DQE  Data  Quality  Baseline  Assessment  Report.  (See  Appendix  B  for  the 
content  outline  of  a  Data  Quality  Baseline  Assessment  Report.) 


4.  Mpni^r,  jTm>igm.gnauon  and  Confflivg  Action  Progress-  The  dqe  team  must 
monitor  implementation  progress  of  all  improvement  opportunities  and  corrective  actions  to  data 
values.  The  Data  Trouble  Reports  (DTRs)  are  useful  for  monitoring  corrective  actions, 
documenting  what,  when,  and  by  whom  corrective  actions  were  taken  or  documenting 
explanations  of  why  the  corrective  actions  were  not  taken.  An  improvement  opportunity  may 
take  longer  to  implement  than  the  time  allotted  for  the  DQE  project  due  to  the  time  needed  to 
get  plans  approved  and  to  allocate  resources.  In  this  case,  the  DQE  team  may  assign  individuals 
to  follow  up  on  the  improvements  periodically.  Although  implementing  the  solution  is  near  the 
end  of  the  DQE  project,  the  DQE  team  must  also  continue  to  monitor  the  solution  to  ensure  it 
remains  effective.  Thir  is  why,  future  data  collection  and  measurement  techniques  must  be  built 
into  the  implementation  plan.  Also,  eventually,  specific  quality  assurance  activities  will  be 
eliminated  from  the  functional  processes  due  to  improvements,  and  the  overall  quality 
improvement  cycle  will  continue  forever,  without  end. 


4-26 


February  4,  1994 


-  769  - 


DoD  S320.1-M-3 


5  Validate  and  riocumcnt  Improved  Data  Quality.  The  DQE  team  must  reassess  the  data 
to  determine  how  well  the  actual  performance  matches  the  planned  improvements.  Tracking  the 
effectiveness  of  improvement  efforts  is  actually  considered  a  part  of  the  measurement  and 
analysis  efforts.  It  includes  re-calculating  the  metrics  to  validate  data  quality  improvement  results 
to  establish  a  new  data  quality  baseline  and  calculating  a  new  PQC  baseline  to  show  a  reduction 
in  costs  and  a  return  on  investments  for  data  quality  assurance.  After  the  improvement  and/or 
corrective  actions  are  validated,  the  DQE  team  should  document  any  improved  data  quality, 
reduced  costs  and  the  DQE  performance  data  thoroughly  in  the  Data  Quality  Baseline  Assessment 
Report.  But.  often  times  the  new  source  data  values  cannot  be  obtained  until  after  the 
improvement  opportunity  has  had  ample  time  to  be  affective.  Sometimes  this  time  period  is 
longer  then  the  planned  DQE  project  timeframe.  In  this  case,  the  DQE  team  should  conduct  a 
trial  implementation  on  the  suspect  data  to  simulate  the  effect  the  improvement  will  have  overall. 
New  or  predicted  baselines  should  be  used  to  justify  making  the  improvement  solution  permanent 
and  for  recommending  follow-up  actions  or  subsequent  improvement  efforts.  Documenting  data 
quality  improvements  allows  others  to  benefit  from  the  lessons  learned  and  gr.is  recognition  for 
data  quality  assurance  and  DAdm. 


G.  VALUE  OF  THE  DoD  DOE  METHODOLOGY 

1 .  Past  results  have  shown  that  the  DoD  DQE  methodology  can  be  used  to  improve  data 
quality,  these  improvements  can  be  measured,  and  processes  can  be  put  in  place  to  continuously 
monitor  and  further  improve  DoD  data  quality.  As  a  result,  migration  toward  standards- 
compliant  systems  can  be  simplified  and  the  cost  of  developing  new  systems  or  maintaining 
existing  ones  is  reduced.  One  of  the  important  features  of  the  DoD  DQE  methodology  is  that 
measures  of  effectiveness  are  built  into  the  process  so  it  is  possible  to  measure  what  is  being 
managed. 

2.  Data  quality  engineering  supports  the  data  migration  strategies  outlined  in  the  three- 
phase  functional  management  process  strategy  for  the  management  of  DoD  information  (DoD 
8320. 1  -M.  reference  (q)).  Phase  1  is  the  establishment  of  a  functional  architecture  and  a  strategy 
for  meeting  functional  requirements.  Phase  2  is  the  establishment  of  baselines  for  process,  data, 
and  information  systems.  This  phase  entails  selecting  information  systems,  which  are  then 
designated  'migration  systems."  to  support  existing  business  processes.  Phase  3  is  the 
improvement  of  functions,  data,  and  information  systems.  This  phase  emails  business  process 
reengineering  and  information  technology  enhancement. 
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3.  The  products  from  the  DoD  DQE  methodology  support  both  Phase  2  and  Phase  3  of 
the  functional  management  process.  In  Phase  2.  the  data  quality  baselines  are  used  to  measure 
migration  potential  for  legacy  applications  and  selecting  migration  systems.  In  Phase  3,  the 
products  of  the  DQE  methodology  are  used  to 

a.  identify  opportunities  for  functional  process  improvement. 

b.  improve  the  quality  of  existing  legacy  data, 

c.  build  data  quality  requirements  into  new  target  systems, 

d.  identify  the  best  data  source  from  which  to  populate  target  systems,  and 

e.  develop  a  migration  strategy  for  convening  legacy  data  values  to  standard 
data  structures. 


4.  The  enforcement  of  data  quality  standards  using  DQE  is  vital  for  establishing  and 
maintaining  the  credibility  of  DoD  data..  Poor  quality  data  can  be  an  expensive  deterrent  to 
cross-functional  and  cross-system  integration. 
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TOOLS  AND  TECHNIQUES 


A.  INTRODUCTION 


1.  This  section  identifies  tools  and  techniques  for  the  detection  and  correction  of  data 
errors.  These  tools  and  techniques  support  ths  Total  Data  Quality  Management  (TDQM)  process 
and  the  DoD  Data  Quality  Engineering  (DQE)  methodology.  Also,  this  chapter  will  address 
detection  and  correction  techniques  to  be  done  in  the  analysis  phase. 


B.  TOOLS  AND  TECHNIQUES 


1.  This  section  provides  additional  guidance  on  the  use  of  the  generic  tools  and 
techniques  that  were  identified  to  support  the  TDQM  process  and  the  DoD  DQE  methodology. 
They  are  representative  of  the  tools  that  are  used  to  improve  any  process  and  are  presented  here 
to  provide  awareness  of  the  different  tools  and  techniques  available,  when  to  use  them,  and  how 
to  use  them.  Other  sources  should  be  consulted  to  obtain  in-depth  information  concerning  these 
tools  and  techniques. 


2.  There  are  a  number  of  factors  that  may  lead  to  the  decision  for  using  a  particular  tool 
or  set  of  tools,  including  the  individual/DQE  team  experience  and  preferences.  Most  tools  are 
oriented  to  a  certain  type  of  activity,  as  identified  in  the  previous  Chapters  of  this  Manual. 
Table  5-1  summarizes  the  identified  tools  and  techniques  to  support  the  TDQM  process, 
including  the  DoD  DQE  methodology,  and  the  Steps  in  which  the  tools  and  techniques  are  used. 
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DoD  DQE  Methodology 


Tools  Si  Techniques 


1.  DDRS  _ _ 


2.  Data  Selection  Evaluation 


3.  Data  Model 


4.  Activity  Model 


5.  Nominal  Group 
Technique 


6.  Cost  Of  Quality 


7.  Quality  Function 
Deployment 


8.  Benchmarking 


9.  Control  Chart 


10.  Data  Trouble  Report 


1 1.  Pareto  Chart 


12.  Histograms 


13.  DQE  Automated  Tool 


14.  Checksheets 


15.  Cause  &.  Effect 


16.  Process  Flow  Analvsis 


Table  5-1  DoD  TDQM  Tools  and  Techniques 
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1.  r>r»n  Defense  Data  Repository  System  (DDRS).  The  DoD  DDRS  was  implemented 
in  the  Department  of  Defense  in  response  to  the  DoD  Directive  8320.1,  Data  Administration, 
dated  September  26,  1991  (reference  (a)).  DoD  Directive  8320.1,  Data  Administration, 
mandates  the  development,  operation,  and  maintenance  of  a  repository  system  as  a  primary  tool 
of  DoD  DAdm.  The  DDRS  is  a  centralized  repository  of  information  about  DoD  data  which  is 
accessible  to  DoD  Components  and  users.  It  is  a  centrally  controlled  DoD-wide  data  repository 
that  manages  and  stores  information  about  data  such  as  meaning,  relationships  to  other  data, 
origin,  usage,  and  format.  Additionally,  the  DoD  DDRS  is  th-  tool  used  to  standardize  data. 


a.  There  are  specifically  documented  objectives  for  the  DDRS.  These  objectives 
state  that  the  DDRS  collects  and  stores  information  about  DoD  Standard  Data  Elements,  Generic 
Elements,  and  Non-Standard  Data  Elements  (migration  data  elements). 


b.  This  information  includes  data  about  each  of  these  elements.  Data  includes  the 
element's  metadata  (characteristics  of  the  element).  In  addition  to  metadata  about  the  element, 
the  DDRS  also  maintains  information  on  users  of  the  DDRS  as  well  as  information  on  users  of 
DoD  Standard  Data  Elements. 


c.  Another  objective  of  the  DDRS  is  to  track  and  monitor  the  status  of  an  element 
(DoD  Standard  Data  Element  or  Generic  Element)  through  its  standardization  phases  from 
development  to  an  approved  element  to  an  archived  element.  All  modifications  made  to  an 
dement  during  its  standardization  phases  are  maintained  and  stored  in  the  DDRS. 


d.  A  third  objective  of  the  DDRS  is  to  use  it  as  a  standardization  tool  to  progress 
the  Standard  Data  Element  and  Generic  Element  through  the  standardization  phases. 


e.  A  fourth  objective  is  to  provide  the  DDRS  as  a  DoD-wide  accessible,  on-line 
standardization  tool  for  use  in  creating  and  modifying  elements,  and  then  querying  and  reporting 
on  data  stored  in  the  DDRS.  As  the  DDRS  becomes  more  accessible  to  the  DoD  user 
community,  it  will  become  increasingly  more  valuable  to  the  DoD  data  quality  assurance 
program  for  defining  data  elements.  If  there  is  a  new  data  requirement,  the  DDRS  Develop 
Element  functions  are  used  to  develop  new  Standard  Data  Elements  and/or  Generic  Elements. 
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2.  Data  Selection  Evaluation  Technique.  When  identifying  a  data  quality  project,  many 
criteria  should  be  reviewed  to  determine  the  suitability  of  the  project.  Prior  to  the  evaluation 
and  selection  of  data  for  data  quality  improvement  (whether  an  initial  effort  or  a  prototype  effort) 
a  set  of  selection  criteria  roust  be  developed.  Major  evaluation  areas  should  be  identified  and 
selection  criteria  are  to  be  developed  for  each  area.  The  evaluation  areas  should  focus  on  the 
availability  of  information  needed  for  the  DoD  DQE  process  detailed  in  Step  3.  Criteria  and 
weighting  factors  should  be  applied  to  evaluate,  select,  and  perform  the  DoD  DQE  analysis. 
The  score  for  each  criteria  should  range  from  0  to  10,  with  the  highest  scores  indicating  the  data 
that  best  meets  the  preferred  outcome  for  each  criteria.  The  evaluation  areas  should  address: 
accMs  to  data;  documentation  completeness;  adequacy;  availability;  access  to  author;  access  to 
functional  and  technical  data  experts,  and  system  data  quality  history  ("Data  Quality  Engineering 
(DQE)  Pilot  Project,  Technical  Report,  reference  (p)).  Examples  of  Data  Selection  Criteria 
evaluation  factors  to  be  considered  are  developed  Li  Table  5-2. 


a.  It  is  critical  to  evaluate  the  system  and  identify  the  system  as  a  legacy  or 
migration  system.  This  will  determine  what  stage  of  the  AIS  life-cycle  data  quality  improvement 
development  efforts  will  occur. 
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Data  Selection  Criteria 

|  Criteria 

Weight 

Description  || 

||  Access  To  Data  | 

Media  for  Download 

Is  media  for  data  download  tape,  disk,  or  electronic? 

Is  media  conversion  required  to  use  data  with  DoD 

DQE  tool?  Is  data  formatted  in  ASCII  or  a  database 
format?  1600  bpi  tape  in  ASCII  format  preferred. 

Existing  Download 
vs.  Custom  Download 

Is  there  an  existing  extract  program  or  operating 
system  utility  available  fcr  download  or  must  a 
custom  download  program  be  developed?  No 
program  development  preferred. 

Unrestricted  Access  to 
Data 

Are  there  restriction  on  the  provision  or  use  of  the 
data?  No  restrictions  are  preferred. 

Cost  of  Data  Retrieval 

Is  there  cost  associated  with  the  download  and 
provision  of  the  data?  No  cost  is  preferred. 

Authoritative  Source 
Identifiable 

Is  there  an  authoritative  source  for  metadata  and  data 
correctness  for  use  in  resolving  conflicts  in  data 
definitions  and  values  during  DoD  DQE  analysis? 

An  identifiable  and  available  authoritative  source  is 
preferred. 

Nonclassified  / 
Nonsensitive 

Is  the  data  classified  or  sensitive?  Nonclassified  and  | 
nonsensitive  data  is  preferred.  | 

Data  Well-structured  - 
No  Multiple 
Concepts/Uses 

Are  the  data  structures  clearly  defined  and  used 
consistently  across  the  applications?  Single  use 
concepts  and  clearly  defined  structures  are  preferred. 

Table  5*2  DoD  DQE  Data  Selection  Criteria 
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Data  Selection  Criteria 


Criteria 


Weight 


Description 


Current  Documentation  (completeness,  adequacy,  availability,  author) 

(For  all  documents  current,  complete,  well  structured,  available 
documentation  with  access  to  the  document  author  is  preferred.) 


Data  Element 
Dictionary  (DED) 


Database  Specification 


Data  Model  (logical 
and  physical) 


Business  Rules 


Process  Model 


System  Functional 
Description 


System  Design 
Specifications  Author 


.  Orders  and 
Regulations 


What  is  the  status  and  contents  of  the  DED? 


What  is  the  status  of  the  Database  Specification  for 
the  system? 


What  is  the  status  of  the  logical  and  physical  data 
models  for  the  system? 


What  is  the  status  of  the  business  rules  that  apply  to 
the  processing  of  the  system  data? 


What  is  the  status  of  the  process  model  for  the 
activities  that  he  system  supports? 


What  is  the  status  of  the  system  functional 
description? 


What  is  the  status  of  the  system  design 
specifications?  Is  the  author  available  for 
consultation? 


What  is  the  status  of  the  orders  and  regulations  that 
apply  to  uie  definition  and  use  of  the  system  data? 


Table  5-2  DoD  DQE  Data  Selection  Criteria  (continued) 
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Data  Selection  Criteria 


i 


Criteria 

|  Weight 

Description 

ii  - 

II  Access  To  functional  Experts 

Location  of  Experts 

Are  system  functional  experts  within  the  local  travel 
area  or  is  long  distance  travel  required  to  obtain  face- 
to-face  support  when  necessary?  Local  area  is 
preferred. 

Willingness  to 
Cooperate 

Are  the  functional  experts  available  during  the  period 
of  performance  for  the  DoD  DQE  project?  Is  there  a 
willingness  on  the  part  of  the  functional  experts  and 
their  organizations  to  participate  in  the  DoD  DQE 
project?  Functional  expen  availability  and 
willingness  is  preferred. 

Political  Roadblocks 

Are  there  any  obstacles  in  obtaining  or  that  would 
delay  participation  by  functional  experts,  such  as 
organizational,  financial,  or  contractual,  policy 
conflicts?  No  obstacles  are  preferred. 

|  Access  To  Technical  Experts 

Location  of  Experts 

Are  system  technical  experts  within  the  local  travel 
area  or  is  long  distance  travel  required  to  obtain  face- 
to-face  support  when  necessary?  Local  area  is 
preferred. 

Willingness  to 
Cooperate 

Are  the  technical  experts  available  during  the  period 
of  performance  for  the  DoD  DQE  project?  Is  there  a 
willingness  on  the  part  of  the  technical  experts  and 
their  organizations  to  participate  in  the  DoD  DQE 
project?  Technical  expert  availability  and  willingness 
is  preferred. 

Political  Roadblocks 

Are  there  any  obstacles  in  obtaining  or  that  would 
delay  participation  by  technical  experts,  such  as 
organizational,  financial,  contractual,  or  policy 
conflicts?  No  obstacles  are  preferred. 

Table  5-2  DoD  DQE  Data  Selection  Criteria  (continued) 
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Data  Selection  Criteria 

Criteria 

Weight 

Description  1 

Data  Background 

Historical  Data 

Quality  Problems 

Are  there  any  known  data  quality  problems 
associated  with  the  systems  that  would  assist  in 
focusing  the  DoD  DQE  analysis?  Clearly  identified 
historical  data  problems  are  preferred. 

Criticality  to 
Organization 

How  critical  is  the  systems  and  its  data  to  the  using 
organization?  Organizations  with  highly  critical  data 
would  be  expected  to  have  greater  interest  in 
participating  in  the  DoD  DQE  project.  Also,  highly 
critical  systems  would  be  expected  to  have  more 
rigorous  data  edits.  Highly  critical  data  is  preferred. 

Opportunity  to 
Demonstrate  DoD 

DQE  Methodology  in 
a  New  Environment 

Is  the  DoD  DQE  methodology  being  demonstrated  on 
a  non-Marine  Corps  system?  DoD  DQE  was 
developed  in  the  Marine  Coip  environment. 
Demonstration  of  the  DoD  DQE  methodology  on  a  | 
non-Marine  Corps  system  is  preferred.  g 

Data  Environment  f 

Government 

Ownership 

Is  the  system  and  the  data  owned  by  the  Government 
or  are  there  contractor  proprietary  rights  on  the 
system  or  data?  Fully  Government  owned  systems 
and  data  are  preferred. 

Can  Be  Scoped  to 
"Double"  Size 

Are  the  data  structures  well-defined  and  segmentable 
so  that  the  DoD  DQE  project  can  be  sized  to  meet 
the  time  restraints?  Well-defined,  segmentable  data  1 
structures  are  preferred.  (For  prototype  efforts,  the  1 
effort  should  take  no  more  than  30-60  days  to  | 

complete.)  | 

Table  5-2  DoD  DQE  Data  Selection  Criteria  (continued) 
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Data  Selection  Criteria 

Criteria 

Weight 

Description  || 

j  Data  Environment  (continued) 

Good  Cost  Reporting 
-  Can  Identity  and 
Reduce  Quality  Costs 

Can  the  quality  or  lack  of  quality  of  the  syst^.-is  data 
be  sufficiently  defined  in  financial  terms  to  show  the 
potential  cost  savings  that  could  result  from  a  DoD 

DQE  analysis  of  the  entire  system?  Clear 
identification  of  the  value  of  quality  data  is  preferred. 

Opportunity  for  Other 
Metrics  Assessment 

Does  the  system  and  data  offer  the  opportunity  to 
define  additional  data  quality  metrics  or  expand  the 
applicability  of  DoD  DQE?  Expansion  opportunities 
are  preferred. 

Active  Data 
Administration 

Program 

Is  there  an  active  data  administration  program  in  the 
organization  that  manages  and  supports  the  system? 

An  active  data  administration  program  is  preferred. 

Low  Risk 

What  is  the  risk  of  meeting  the  objectives  of  the  DoD 
DQE  project  within  the  allotted  schedule  and  cost 
using  the  designate  system?  Low  risk  is  preferred. 

High  Payoff 

What  is  the  potential  level  of  benefit  of  conducting 
the  DoD  DQE  methodology  on  the  designated  system 
and  the  system  owner?  High  level  of  benefit  is  1 

preferred.  | 

Table  5-2  DoD  DQE  Data  Selection  Criteria  (continued) 


3.  Data  Model.  A  data  model  is  defined  as  "in  a  database,  the  user’s  logical  view  of  the 
data  in  contrast  to  the  physically  stored  data,  or  storage  structure;  a  description  of  the 
organization  of  data  in  a  manner  that  reflects  the  information  structure  of  an  enterprise"  (DoD 
8320. 1-M,  reference  (d)). 
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4.  Activity  Model.  An  activity  model  is  defined  as  "models  of  the  processes  that  make 
up  the  functional  activity  showing  inputs,  outputs,  controls,  and  mechanisms  through  which  the 
processes  of  the  functional  activity  are  (or  will  be)  conducted"  (DoD  8320. 1-M,  reference  (d)). 


5.  Nominal  Group  Technique  fNGTl.  The  NGT  is  a  good  team  development  tool 
especially  with  new  teams  that  may  be  cautious  about  disclosing  opinions  and  ideas.  The  NGT 
is  a  variation  of  "brainstorming"  and  generates  ideas  and  creates  team  energy.  A 
"brainstorming"  activity  helps  unlock  the  creativity  of  a  group  and  is  more  effective  than  trying 
to  generate  ideas  as  individuals.  Brainstorming  is  a  group  activity  that  is  used  to  stimulate  ideas, 
such  as  possible  problems,  causes  of  problems,  or  solutions  to  problems.  The  NGT  is  a 
variation  of  brainstorming.  In  brainstorming,  ideas  are  generated  aloud  through  a  round-robin 
technique  ("Teams  for  Excellence:  Skills,  Strategies  &  Implementation  Exercise  Book", 
reference  (r)).  In  the  NGT,  the  group  generates  individual  lists  first  and  then  combines  the  lists. 


a.  More  specifically,  the  guidelines  for  performing  the  NGT  include: 


-  Identify  a  Topic 

-  Each  Team  Member  Creates  A  List  Of  Ideas 

-  Each  Team  Member  Reads  The  List  Aloud  To  The  Group 

-  A  Master  List  Of  Ideas  Is  Created 

-  The  Group  Analyzes  The  Master  List 

-  Condense  and  Rank  The  List  By  Voting 

-  The  Team  Takes  Action  On  The  Highest-Ranking  Items  (Or  Implements 
The  Highest  Ranking  Ideas 


.Cost  Of  .Qualm-  A  data  quality  baseline  is  needed  as  a  starting  point  for  data  quality 
improvement.  A  baseiinc  is  a  set  of  information  metrics  expressed  in  some  form  or  model. 
Activities  are  the  building  blocks  of  buildn.g  data  quality  assurance.  Therefore,  it  is  essential 
to  understand  these  activities  to  implement  data  quality  improvement.  The  technique  used  in 
DoD  for  looking  at  activities  and  understanding  them  is  called  Activity  Based  Costing  (ABC). 
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Through  the  use  of  ABC,  activities  for  data  quality  can  be  defined  (activities  under  the  categories 
of  Prevention,  Appraisal,  Internal  Failure  and  External  Failure)  and  costs  based  on  or  associated 
with  these  activities  can  be  determined.  ABC  shows  managers  what  they  do  with  their  money 
("Corporate  Information  Management,  Process  Improvement  Methodology  For  DoD  Functional 
Managers’,  reference  (t)). 


BEFORE 

Prevention 

Appraisal 


Internal 

Failures 


External 

Failures 


AFTER 


Prevention 


Appraisal 

Internal 

Failures 

External  Failures 


Figure  5-1  Cost  of  Quality 


a.  This  ability  to  match  costs  with  activities  quickly  highlights  where  improvement 
is  needed,  whether  one  is  determining  long-term  data  quality  improvement  priorities,  or 
measuring  short-term  success.  Through  ABC,  functional  managers  can  characterize  the  value 
of,  or  need  for,  each  activity.  In  turn,  these  characterizations  can  be  used  to  rank  activities  for 
improvements. 
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b.  In  general,  creating  a  cost  baseline  includes  first  identifying  the  activities  that 
apply  to  managing  data.  Second,  identify  the  costs  of  data  quality  that  are  associated  with  the 
different  activities  that  apply  to  managing  the  data. 


c.  More  specifically,  first  identify  all  of  the  quality  cost  elements,  under  each 
category,  related  to  the  data  under  analysis.  Once  the  cost-elements  list  is  completed,  each 
element  should  be  classified.  Costs  must  first  be  differentiated  between  operational  (business) 
costs  and  costs  of  quality.  Costs  of  quality  are  identified  as  being  Prevention  Costs,  Appraisal 
Costs,  Internal  Costs,  and  External  Costs. 


7.  Quality  Function  Deployment  (QFD).  The  QFD  technique  is  a  conceptual  map  that 
provides  the  means  for  cross-functional  planning  and  communications.  It  is  a  method  for 
transforming  customer  wants  and  needs  into  quantitative,  engineering  terms.  When  developing 
a  QFD,  the  following  questions  need  to  be  asked:  (1)  What  data  attributes  are  desired?,  (2)  Are 
all  attributes  equally  important?,  (3)  What  are  the  engineering  characteristics  that  match  the  data 
attributes?,  (4)  How  does  each  engineering  characteristic  affect  each  data  attribute?,  and  (5) 
How  does  one  engineering  change  affect  other  characteristics?  (DoD  5000.5 1-G,  reference  (o)). 


Figure  5-2  Quality  Function  Deployment 
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8  Benclmiadgllg-  Benchmarking  is  a  method  of  measuring  processes  against  those  of 

recognized  performance.  It  helps  to  establish  priorities  and  targets  leading  to  process 
improvement.  By  understanding  where  the  current  level  of  performance  is  in  comparison  to 
other  identified  levels  of  performance  allows  for  gauging  performance.  When  performance  is 
measured  against  other  -benchmarks"  various  processes  for  improvement  can  be  targeted  (DoD 
5000.51-G,  reference  (o)). 


9.  Cflpgal  Charts.  Control  Charts  are  a  graphic  representation  of  measured  actual 
performance  relative  to  computed  control  limits.  They  are  used  to  show  the  variation  on  process 
variables  and  identify  special  causes.  This  technique  allows  the  distinction  between 
measurements  that  are  predictably  within  the  inherent  capability  of  the  process  (common  causes 
of  variation)  and  measurements  that  are  unpredictable  and  produced  by  special  causes  (DoD 
5000.51-G,  reference  (o)). 


Figure  5-3  Control  Charts 


10.  DataTrouble  Report  (DTR).  A  DTR  includes  creating  a  standard  form  for 
documenting  and  improving  data  quality  problems.  The  form  should  include  the  following 
information  topics:  (1)  Define  The  Problem,  (2)  Analysis  of  Problem,  (3)  Determine  Causes, 
(3)  Action  Taken  to  Eliminate  Cause,  (4)  Validating  The  Solution,  and  (4)  Improvement 
Recommendations. 
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11.  Pareto  Chart.  A  Pareto  Chart  is  a  special  form  of  vertical  bar  graph  which  helps 
determine  which  problems  to  solve  in  what  order.  This  bar  graph  arranges  bars  in  descending 
order,  with  the  largest  to  the  left.  Each  bar  represents  a  problem.  The  chart  displays  the 
relative  contribution  of  each  sub-problem  to  the  tool  problem.  Doing  a  Pareto  Chart  based  upon 
either  Check  Sheets  or  other  forms  of  data  collection  helps  direct  attention  and  efforts  to  the 
important  problems.  In  general,  more  is  gained  from  working  on  the  tallest  bar  (e.g.,the  most 
detected  data  errors)  than  on  the  smaller  bars.  The  basic  steps  on  creating  a  Pareto  Chart  are: 
(1)  Select  the  problems  that  are  to  be  compared  and  rank  order,  (2)  Select  the  standard  for 
comparison  unit  of  measurement,  (3)  Select  time  period  to  be  studied,  (4)  Gather  the  necessary 
data  on  each  category,  (5)  Compare  the  frequency  or  cost  of  each  category  relative  to  all  other 
categories,  (6)  List  the  categories  from  left  to  right  on  the  horizontal  axis  in  their  order  of 
decreasing  frequency  or  costs,  (7)  Above  each  classification  or  category,  draw  a  rectangle  whose 
height  represents  the  frequency  or  cost  in  that  classification  (DoD  500C.51-G,  reference  (o)). 


Figure  5-4  Pareto  Chart 
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12.  Histograms.  A  Histograms  is  a  graph  that  displays  frequency  of  data  in  a  column 
form.  A  Histogram  takes  measurement  data  and  displays  its  distribution.  Tnis  help-,  to  identify' 
changes  or  shifts  in  data  or  in  processes  as  changes  are  made.  It  shows  how  variable 
measurements  of  a  process  or  data  errors  can  be,  and  it  helps  in  the  establishment  of  baselines. 
Once  baselines  have  been  set,  measurements  can  be  compared  to  these  standards.  The  basic 
steps  for  construction  a  Histogram  are:  (1)  Determine  the  measurements  to  be  taken,  (2)  Collect 
data,  (3)  Organize  data  into  incremental  units,  and  (4)  Develop  a  graph  to  pictorially  summarize 
the  results  (DoD  500G  51-G,  reference  (o)). 


Figure  5-5  Histogram 
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13.  POE  Automated  Tool.  The  DQE  automated  tool  is  used  to  (1)  assess  the  accuracy, 
validity,  and  usefulness  of  actual  data  values,  and  to  (2)  trigger  the  DAdm  process  for  data 
validation  and  business  process  correction.  The  standards  information  derived  during  data 
analysis,  modeling,  and  standardization  is  used  to  develop  automated  rule  sets  to  check  actual 
data  values  and  associated  metadata  for  accuracy  and  consistency.  For  example,  standard  code 
tables  are  used  as  the  foundation  for  domain  checks,  and  the  relationships  defined  in  data  models 
are  translated  into  operational  rule  sets.  The  DQE  tool  performs  zero  and  blank  checking, 
mathematical  calculations,  operational  rule  set  validation,  and  statistical  tests  and  validation  to 
check  data  values  for  accuracy  and  consistency.  The  DQE  tool  permits  continuous  assessment 
of  the  progress  made  toward  aciiieving  total  data  quality  ("Data  Quality  Engineering  (DQE)  Pilot 
Project,  Technical  Report,  reference  (p)). 


a.  In  the  DQE  methodology,  data  analyses  is  performed  on  individual  data 
elements  to  determine  the  quality  requirements.  The  requirements  are  used  to  generate  rule  sets 
which  provide  the  foundation  for  specific  data  quality  engineering  routines  to  be  constructed 
within  a  data  quality  tool.  A  data  quality  tool  executes  the  routines  to  perform  data  quality 
assessments  on  each  data  element  analyzed.  Some  types  of  data  quality  checks  performed  by 
tools  are:  searches  for  unqualified  blank  or  zero  data  values,  duplicate  records  (based  on  key 
field  comparisons),  and  invalid  domains/ranges.  Checks  are  performed  to  detect  errors  by 
comparing  a  given  data  value  to  a  calculated  value  using  related  data  in  a  given  record  or 
database.  Also,  data  values  are  subject  to  checks  based  on  operationally  oriented  (if... then)  rule 
sets. 


b.  A  data  quality  tool  should  be  "data-driven"  to  allow  for  easy  modification  to 
work  with  virtually  any  type  of  data  or  any  combination  of  databases  and  to  perform  any  type 
ot  quality  check.  It  should  also  have  the  capability  to  be  configured  and  controlled  without  the 
support  from  programmers  or  system  analysts.  The  tool  should  produce  a  definitive  list  of  data 
errors  and,  often  predict  correct  data  values.  The  tool  should  automatically  find  and  flag  data 
errors  and  include  a  means  for  tracking  data  error  corrections  and  providing  the  DAd  with  a 
means  of  automatically  generating  metric  reports  in  graphic  and  table  formats. 
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14.  Chcgtehects  also  facilitate  data  collection  by  providing  a  standardized  format  for 
recording  information.  Checksheets  include  a  list  of  check-off  items  that  permit  data  to  be 
collected  quickly  and  easily  in  a  simple  standardized  format  that  lends  itself  to  quantitative 
analysis.  Checksheets  are  frequently  used  to  collect  data  on  the  number  of  defects/errors 
identified.  Checksheets  are  conducted  by  (1)  Identifying  the  data  to  be  collected,  (2)  Designing 
a  Checksheet  to  collect  data,  (3)  Collecting  the  data,  and  (4)  Tabulating  results  (DoD  S000.S1-G, 
reference  (o)). 


Checksheet 

Product:  Haceivar  unit  XYZ  Date:  0/Q9/B9 
Total  examined:  2QQ  Name-  Smith 


jKaESs ? 

*. ...1.  -  y  .....  j.  . i 


/ ' ' 'W.  «//.  "‘'Sir/.  •  •////// y - .  V 
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Figure  5-6  Checksheet 
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15.  Pause  &.  Effect.  The  cause  and  effect  diagram  was  developed  to  represent  the 
relationship  between  some  'effect"  and  all  of  the  possible  "causes"  influencing  it.  The  effect 
or  problem  is  stated  on  the  right  side  of  the  chart  and  the  major  influences  or  "causes"  are  listed 
to  the  left.  Begin  by  trying  to  pick  a  problem  that  is  controllable  within  the  organization.  The 
basic  steps  in  creating  a  cause  and  effect  include:  (1)  Naming  the  Problem,  (2)  Deciding  on  the 
major  categories  of  causes,  (3)  Brainstorming  for  more  detailed  causes,  (4)  Eliminating  causes 
that  do  not  apply,  (5)  Discussing  the  remaining  causes  and  decide  which  are  most  important.  (6) 
Working  on  the  most  important  causes,  and  (7)  Eliminating  or  Correcting  Causes  (DoD  5000.51- 
G,  reference  (o)). 


Figure  5-7  Cause  &.  Effect 
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16.  Process  Flow  Analysis.  Process  Flow  Analysis  is  a  structured  system  that  documents 
all  the  steps  in  a  work  process  or  procedure  and  will  identify  areas  where  problems  are  prevalent 
and  opportunities  for  improvement.  It  helps  to  improve  a  process  by  isolating  the  causes  of 
problems  or  potential  problems  associated  with  the  process.  It  is  important  to  set  the  boundaries 
of  the  process  that  is  to  be  analyzed,  prior  to  this  exercise.  The  guidelines  for  performing  a 
Process  Flow  Analysis  include  (DoD  5000.5 1-G,  reference  (o)): 


-  Diagram  The  Process 

-  Have  Team  Members  Identify  The  Procedures  and  Potential  Problems  That 
Can  Occur  In  Each  Stage  Of  The  Process 

-  Analyze  The  Process 

•  Redesign  The  Process  By  Incorporating  The  Selected  Ideas  For  Improvement 
|  -  Test  The  Process  Changes 

i 

I  -  Document  The  Process  Changes 


r 
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Figure  5-8  Work  Flow  Analysis 
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The  Data  Resources  Management  and  Standardization  Program  (DRM&S)  began  on  February 
11,  1991.  The  principal  objective  of  the  DRM&S  program  was  to  supply  accurate,  valid,  and 
reliable  data  to  Marine  Corps  logistics  planners.  DRM&S  developed  a  Data  Quality  Engineering 
(DQE)  program  as  a  means  of  enhancing  the  quality  of  source  data  files  used  by  the  Marine  Air 
Ground  Task  Force  Logistics  Automated  Information  Systems  (MAGTF  H/LOG  AIS)  family  of 
PC-based  logistics  systems,  used  for  planning,  deployment,  employment  and  sustainment.  The 
program  continues  to  achieve  this  objective  by  developing  and  enforcing  data  standards,  finding 
and  correcting  data  errors,  and,  finally,  identifying  and  implementing  needed  functional  process 
improvements.  DRM&S  developed  the  DQE  methodology  which  encompasses  the  development 
and  implementation  of  a  scries  of  steps  to  ensure  accurate  data.  DRM&S  used  DQE  to  identify 
data  errors  and  trigger  appropriate  corrections  and  process  changes. 

The  DRM&S  program  was  tasked  to  develop  and  field  a  system  that  facilitated  logistics  data 
administration.  The  DRM&S  developed  the  Marine  Air  Ground  Task  Force  Data  Library 
(MDL)  System  to  meet  that  requirement  and  to  provide  the  means  for  achieving  logistics  data 
distribution  and  control.  The  MDL  also  served  as  the  host  for  the  automated  data  quality 
engineering  tool  used  to  achieve  and  maintain  data  quality. 

The  DQE  tool  iS  the  principal  function  of  the  Marine  Air  Ground  Task  Force  Data  Library 
(MDL)  System.  The  analyses  provided  the  foundation  on  which  specific  data  quality  engineering 
routines  were  constructed  within  MDL.  The  knowledge  acquired  through  these  analyses  allowed 
a  system  user  to  establish  and  modify  the  data  quality  controls  implemented  within  the  MDL 
system.  Because  the  DQE  tool  is  developed  using  object-oriented  code,  the  program  is 
transportable  and  readily  adaptable  to  a  variety  of  data  file  structures. 

Tbs  case  has  been  preoared  to  demonstrate  a  •real-life*  application  of  the  Data  Quality  Engineering  (DQE) 
methodology  to  the  DoD  data  environment  (U.S.  Marine  Corps).  The  DQE  tool  of  the  MDL  system  ts  available 
for  use  throughout  DoD  through  the  Software  Re*Use  Program,  Defense  Software  Repository  System  (DSRS).  In 
1992.  the  DRM&S  were  nominated  for  a  Gold  Nugget  Award  through  the  Defense  Information  Management 
Program. 
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The  DRM&S  implemented  the  DQE  methodology  as  an  integral  part  of  their  Data 
Administration  process  (see  Figure  A-l).  The  U.S.  Maxine  Corps  DQE  methodology  began  with 
a  comprehensive  standardization  process,  supported  by  a  suite  of  custom  designed  CASE  tools. 
The  data  standardization  process  consisted  of  the  following  four  inter-related  analyses,  which 
are  described  in  Chapter  4  of  this  Manual: 

-  High-level  Collision  Analysis 

-  Collision  Analysis 

-  Data  Structure  Analysis 

•  Domain  and  Range  Analysis 


The  MDL  system  automatically  performed  DQE  on  each  data  element  processed.  Data  quality 
routines  included  checks  for  unqualified  blank  or  zero  data  values,  duplicate  records  (based  on 
key  field  comparisons),  and  invalid  domains/ranges.  When  applicable,  a  data  quality  routing 
check  for  errors,  by  comparing  a  given  data  value  to  a  calculated  value  using  related  data  in  a 
given  record  or  database,  was  performed.  Finally,  data  values  were  subject  to  checks  based  on 
operationally  oriented  (if... then)  rule  sets.  MDL  produced  a  definitive  list  of  data  errors  and, 
often,  predicted  correct  data  values.  The  MDL  systems  automatically  found  and  flagged  data 
errors  and  triggered  appropriate  action  from  the  DAd.  Further,  DQE  often  identified  a  probable 
correct  value  and,  frequently,  the  likely  source  or  causes  of  the  problem.  Finally,  the  MDL 
system  included  a  means  for  tracking  data  error  corrections  and  provided  the  DAd  with  a  means 
of  automatically  generating  progress  report  "metrics"  in  graphic  and  table  formats. 

Since  the  initial  implementation  of  DQE  in  DRM&S,  DQE  has  become  an  increasingly  important 
component  of  the  Marine  Corps  logistics  data  administration  process.  The  DQE  effort  focused 
on  finding  and  correcting  problems  with  data  availability,  accuracy,  validity,  and  reliability.  A 
key  feature  of  DQE  lies  in  the  use  of  preprogrammed,  automated  routines  to  check  data  values. 
Data  value  check  routines  vary  in  sophistication  from  a  simple  check  for  a  missing  or  an 
inappropriate  null-data  value  to  use  of  a  complex  rule  set.  Using  DQE  principles,  suspect  data 
was  quickly  identified,  categorized,  and  provided  to  DAds  in  a  form  that  facilitated  action  by 
functionally  oriented  process  action  teams.  In  summary,  DQE  replaced  vague  complaints  about 
"bad  data'  (with,  at  best,  isolated  examples)  with  a  means  of  not  only  capturing  specific  errors, 
but  triggering  action  to  correct  both  the  data  and,  when  appropriate,  a  flawed  process  that 
resulted  in  the  data  error. 
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Figure  A-l  DRM&S 

Data  Administration  Process 
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To  date,  the  DRM&S  program  has  identified  over  8,000  specific  daa  amrs  in  ^logisocs  daa 
from  legacy  systems.  Among  many  other  statistics  the  DQE  process  revealed  that  95  %  of  errors 
were  caused  by  faulty  processes  or  procedures  and  not  dam-entry  problems  or  computer/pi  ogram 
malfunctions.  By  applying  sound  daa  administration  practices,  Marine  Corps  logistics  planners 
have  realized  a  33%  increase  in  the  availability  of  accurate  technical  daa,  with  further 
improvement  occurring  with  each  new  data  release.  Finally,  benefits  of  the  DRM&S  program 
included: 

•  Accurate,  reliable,  valid  data. 

.  •Fiifflinarim  of  multiple  sources  for  similar  data. 

-  Simplified  data  distribution  schemes  and  improved  daa  availability. 

•  Improved  functional  processes  and  procedures. 


Sample  Program  Objectives  and  Accomplishments 

The  following  three  samples  of  weak  performed  under  the  Daa  Resources  Management  and 
Standardization  program  have  been  selected  to  illustrate  the  success  of  the  DQE  effort. 


Sample  1  -  New  Standards  for  Data  Quality 

Prior  to  the  Data  Resources  Management  and  Standardization  (DRM&S)  program,  the  Marine 
Corps  logistics  DAd  was  aware  that  problems  existed  in  data  quality.  While  specific  examples 
of  “bad  data"  were  not  difficult  to  find  the  DAd's  most  pressing  need  was  to  establish  an 
effective  procedure  for  attacking  all  data  errors  and,  where  appropriate,  the  faulty  daa 
administration  process.  "The  DRM&S  program  provided  the  Marine  Corps  DAd  with  the  tools 
needed  to  quantify  daa  errors  and  to  focus  corrective  action. 

Figure  A-2  depicts  a  typical  daa  quality  engineering  product.  This  is  an  example  analysis  of 
six  fields  in  the  Item  Description  File  (IDF),  an  important  Marine  Corps  source  file.  As  shown, 
the  DQE  process  provides  a  unique  insight  into  the  daa  in  the  IDF.  In  this  case,  the  DQE 
process  included  routhr  that: 

Reported  the  percenage  of  valid  (not  blank  or  zero)  values  in  selected  fields. 
Compared  reported  square  and  cubic  measurement  daa  with  a  calculated  value 
based  on  dimensional  daa  stored  in  other  fields. 
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Though  not  shown  in  this  particular  example,  other  object-oriented  code  blocks  are  provided  as 
part  of  the  DQE  tool.  The  additional  routines  include  the  capability  to: 


Compare  data  values  with  those  in  a  given  "look-up”  table. 

Check  to  ensure  a  data  value  is  within  a  stated  range. 

Apply  business  "if... then"  business  rules,  sometimes  using  other  data  values  in  a 
given  record,  to  locate  and  report  data  errors. 


Percent  Valid  or  Accurate 
100 


■  Goal 

□  Release  1 

■  Initial 


l  1 


CCC  Square  Cube  Dimen  Square  Cube 
Valid  (not  blank  or  zero)  In  calc  range 


Figure  A-2  Corrections  Made  to  IDF  During  3rd  Quarter  1992 


In  addition  to  isolating  specific  errors,  the  DQE  tool  allowed  the  DAd  to  track  progress  toward 
restoring  data  quality.  In  this  example,  the  effort  to  correct  cargo  codes  was  clearly  a  success. 

The  data  quality  engineering  tool  contained  the  capability  to  produce  many  such  "metrics  of 
success". 
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The  knowledge  of  data  and  data  errors  that  results  form  data  quality  engineering  efforts  can  be 
applied  to  improving  a  functional  process.  In  fact,  as  the  following  example  illustrates, 
approximately  95%  of  data  errors  are  the  result  of  a  faulty  functional  process  and  not  the 
product  of  a  computer  malfunction  or  a  error  on  the  pan  of  a  data-entry  clerk. 

Data  quality  engineering  revealed  that  technical  data  (dimensions,  weights,  fuel  consumption 
factors,  etc.)  on  Marine  Corps  equipment  were  incomplete  and  inaccurate.  This  kind  of  specific 
knowledge  permitted  Marine  Corps  analysts  to  focus  and  direct  their  efforts.  In  this  example. 
Marine  Corps  analysts  were  able  to  determine  that  a  specific  set  of  data  problems  existed  on 
vehicle-mounted  shop  sets.  Currently,  such  equipment  is  carried  in  standard  S-250  shelters, 
mounted  on  the  bed  of  vehicles  like  the  high-mobility,  multi-purpose,  wheeled  vehicle 
(HMMWV).  Current  data  files,  however,  have  not  kept  pace  with  equipment  modernization 
efforts.  Data  in  current  Marine  Corps  automated  files  include  technical  data  describing  the  shop 
set  mounted  on  an  obsolete  M1028  chassis.  Technical  data  is  current  on  the  S-250  shelter,  the 
HMMWV,  and  several  configurations  of  shop  sets. 

Further,  the  data  files  reflected  that  various  vehicle/shop  set  configurations  are  possible. 
However,  useful  technical  data  on  current  vehicle/shelter/shop  set  configurations  were  not 
available. 

This  analysis  yielded  a  clear  understanding  of  how  the  process  of  fielding  new  equipment  has 
failed  to  generate  the  data  required  to  satisfy  current  operational  requirements.  The  fault  was 
traced  to  a  lack  of  documentation  in  the  current  Marine  Corps  systems  used  to  enter  and  crack 
this  data  and  corrective  action  are  underway. 


Sample  3  -  Cost  Benefit  of  Data  Quality 

The  impact  of  a  data  error  varies  with  the  magnitude  of  the  error  and  the  way  in  which  the  data 
may  be  used.  Figure  A-3  shows  how  a  simple  data  error  can  result  in  S13.5M  in  additional 
operating  costs!  This  is  an  example  of  an  actual  error  that  had  been  discovered  in  the  Marine 
Corps  Item  Description  File  (IDF).  Among  other  uses,  the  IDF  provides  a  source  for  average 
fuel  consumption  rates.  Rate  data,  in  turn,  is  used  in  war-planning  systems  to  automatically 
calculate  and  aggregate  the  fuel  requirements  of  a  combat  force.  Fuel  requirements  are 
convened  to  strategic  lift  requirements  and  become  a  factor  in  developing  war  plans,  time¬ 
phasing  the  arrival  of  operational  forces  in  a  theater,  and  determining  new  ship-building 
priorities.  In  this  particular  instance,  the  average  fuel  consumption  rate  for  a  light  armored 
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vehicle  (LAV)  was  reported  as  70  gallons  per  hour.  In  fact,  the  actual  rate  was  7  gallons  per 
hour  Because  fuel  consumption  data  are  automatically  calculated  and  aggregated  by  planning 
systems,  this  kind  of  error  easily  goes  undetected.  Applied  data  quality  engineering  routines, 
on  the  other  hand,  just  as  easily  identify  such  errors.  In  this  instance,  a  HQE  routine  designed 
to  flag  fuel  consumption  data  values  not  falling  within  two  standard  deviations  of  the  norm 
captured  this  data  error.  The  potential  additional  costs  resulting  from  this  error 
totaled  S13.5M! 


ttem  | 

Unit  Cost 

Incorrect 

Cost  of 

Rsqmt 

Error 

Fuel  (gal) 

$.074 

1.32M 

$  976,600 

132K 

$97,680 

Refuelers 

$47K 

264 

$  12.400K 

26 

$  1.240K 

Ships  (par 
day  for  15 

$  50K 

12 

$  1.650K 

0.2 

$  165K 

days) 

I 

Totals 

6  15.030K 

$  1,503K 

Error  cost  estimates  basad  on: 


•  LAV  fuel  consumption  of  70  vice  correct  value  of  7  GPH 

-  MEB  deployment  with  60  daye  of  supply 

-  Combat  use  rates  and  std  conversion  factors  (l.e.,  55  gal/barrel) 

-  Refueler  and  fuel  (barrel)  deck  apace  requirements  of  235  and  6.7 
square  feet  respectively 

•  Average  of  100K  squsre  feet  of  storage  per  ship 


Figure  A-3  Additional  Costs  Resulting  From  Inaccurate  Data 
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Executive  Summary 

l.  Introduction 

A.  Background 

B.  Objectives  of  Project 

C.  Project  Description 

D.  Identify  DQF.  team  and  data  experts  who  participated  in  the  project 
II.  DQE  Methodology  (Describe  the  methodology  used) 

m.  Data  Quality  Baseline 

A.  Historical  Data  Problems  and  Types  of  Assessments 

B.  Describe  Data  That  Was  Assessed 

(data  structure,  file  structure,  rule-sets,  etc.) 

C.  Measurement  Method 

(DQE  automated  tool,  manual  methods,  validation  process) 

C.  Results/ Output /Data  Trouble  Reports  • 

D.  Data  Quality  Baseline 

-  Metric  Calculation  results 

-  Graphic  Representations/ Charts 

-  Previous  baselines 

IV.  Root  Cause  Analysis 

A.  Describe  the  Suspect  Data  Analyzed 

B.  Describe  Analysis  Techniques  and  Results 

C.  For  each  root  cause  identified  provide: 

-  Description 

-  Poor-Quality  Cost  Baseline  (graphic  representation/chart) 

-  Recommended  Improvement  Alternatives 
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DATA  QUALITY  BASELINE  ASSESSMENT  REPORT 

(continued) 

V.  Improvement  Results 

A.  Describe  Formal  Notification  Process 

B.  Describe  Improvements 

-  Corrective  actions  taken  to  fix  errors 

-  Implemented  improvement  alternatives 

-  Disposition  of  recommendations  not  implemented 

C.  Provide  Dcta  Quality  Benefits 

-  Improved  data  quality 

-  Reduced  cost 

VI.  Summary  of  Recommendations 


APPENDICES 

May  include: 

-  Raw  Data,  Data  Structures,  Data  models,  Activity  Models 
•  Actual  Rule-Sets 

-  Actual  Reports 

-  Activity  Based  Cost  Data 

-  References 

-  Definitions 
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For  overall  data  administration  responsibilities  refer  to  DoD  S320.1-M  (reference  (d)).  For  the 
purpose  of  the  DoD  Data  Quality  Assurance  Procedures,  these  Roles  and  Responsibilities  have 
been  further  defined  with  respect  to  data  quality  assurance  responsibilities.  The  following  table. 
Table  C-l,  refers  each  role  and  the  corresponding  Chapters  of  this  Manual  that  specifically 
address  the  responsibilities  of  those  most  involved  with  the  data  quality  assurance  process. 
However,  it  is  recommended  that  all  DoD  personnel  familiarize  themselves  with  the  entire 
contents  of  this  Manual  to  gain  a  thorough  understanding  of  the  DoD  Data  Quality  Assurance 
Procedures. 


ROLE 

REFER  TO  SPECIFIED 
CHAPTER  IN 
MANUAL 

A. 

The  Assistant  Secretary  of  Defense  for  Command, 
Control,  Communications,  and  Intelligence  (ASD 
(C3I)) 

Chapter  ? 

B. 

OSD  Principal  Staff  Assistant/ 

Chairman,  Joint  Chiefs  of  Staff  (CJCS) 

Chapter  3 

C. 

Heads  of  DoD  Components 

Chapter  3 

D. 

DoD  Data  Administrator  (DAd) 

Chapters  3,  4,  5 

E. 

Functional  Data  Administration  (FDAd)/ 

Component  Data  Administrator  (CDAd) 

Chapters  3,  4,  5 

F. 

DASD  (IM) 

Chapter  3 

G. 

Director  for  Defense  Information  (DDI )/ 

DDI  Functional  Information  Manager  (FIM) 

Chapter  3 

Table  C-l  Roles  and  Responsibilities 
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ROLE 

REFER  TO  SPECIFIED 
CHAPTER  IN 
MANUAL 

H. 

Database  Administrators  (DBAd) 

Chapters  4,  5 

I. 

Data  Administrators  (DAd) 

Chapters  3,  4,  5 

J. 

Technical  Developers 

Chapters  4,  5 

K. 

Automated  Information  Systems  Program  Managers 
(AIS  PM) 

Chapters  4,  5  f 

m 

Data  Stewards 

Chapters  3,  4,  5 

M. 

Functional  Managers 

Chapters  3,  4,  5 

N. 

Functional  Activity  Program  Manager  (FAPM) 

Chapters  3,  4,  5 

0. 

Program  Managers  (PM) 

Chapters  3,  4,  5 

P. 

Customers/Users  of  Data 

Chapters  3,  4 

Table  C-l  Roles  and  Responsibilities  (continued) 


Specific  Responsibilities 


1.  Data.  Administrators  •  DAds  throughout  DoD  must  manage  the  quality  of  data 
throughout  the  data  life-cycle.  To  support  these  efforts,  DoD  DAdm  has  established  a  Total 
Data  Quality  Management  (TDQM)  process  which  shall  be  used  to  ensure  DoD  data  quality. 
The  DoD  DAd  is  responsible  for  developing  the  long-term  vision,  mission,  and  goals  for  the 
DoD  Data  Administration  Program.  The  DoD  DAd  also  establishes  the  overall  goals  and 
objectives  for  data  quality  assurance.  Each  Functional  Data  Administrator  (FDAd)  and 
Component  Data  Administrator  (CDAd)  prepares  a  strategic  plan  for  their  respective  Functional 
Area  or  Component  in  accordance  with  the  annual  planning  guidance  provided  by  the  DoD  DAd. 
Throughout  DoD,  DAds  and  DBAds  must  incorporate  their  functional  area  or  component  data 
quality  goals  and  objectives  into  their  organization’s  annual  plans.  The  DoD  DAd  will  publish 
annual  reports  on  the  progress  of  DoD  data  quality  assurance.  The  DoD  DAd  should  also 
review  performance  results  against  the  overall  goals  and  objectives  for  data  quality  assurance. 
DoD  DAdm  must  perform  quality  control  activities  to  track  and  document  continuous  process 
improvements  for  DoD  data  quality  assurance. 
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a.  If  required,  the  DoD  DAd,  FDAds,  or  CDAds  will  provide  guidance  and 
assistance  in  the  development  of  these  annual  plans.  By  assessing  the  vision  of  the 
organization's  future  and  coming  to  a  consensus  on  a  data  quality  goal,  the  FDAd,  or  CDAd  will 
solidify  their  understanding  of  the  ultimate  data  quality  goals.  They  must  communicate  that  goal 
throughout  the  organization. 


b.  The  DAd  is  to  make  sure  that  the  functional  manager  is  aware  of  the  DoD  IM 
initiatives  and  data  quality  concerns.  Building  management  awareness  of  the  need  for  change 
and  the  benefits  of  data  quality  improvements  by  the  TDQM  process,  will  help  establish  a 
foundation  for  continuous  data  improvement  efforts.  Awareness  can  be  increased  by  providing 
information  on  the  TDQM  process,  advocating  the  manager's  attendance  at  DoD  DAdm 
conferences  on  data  quality  assurance,  and  inviting  his/her  participation  in  TDQM  activities. 


c.  In  order  to  obtain  top-management  support,  the  DAd  and  other  functional 
experts  may  conduct  a  small,  prototype  effort.  This  effort  should  be  used  to  make  a  case  for 
the  ramifications  of  having  poor  quality  data  and  the  costs  associated  with  poor  data.  These 
assessments  should  be  presented  to  top-management  with  a  supporting  Data  Quality  Plan. 


d.  A  leader  needs  to  be  established  for  th*s  data  quality  function.  This  individual 
will  provide  the  lead  for  establishing  baselines,  performing  assessments,  prioritizing,  providing 
solutions,  and  making  final  improvement  recommendations.  This  role  may  be  performed  by  the 
DBAd,  DAd,  AIS  project  manager,  or  technical  developers.  This  individual  has  the  overall 
operational  responsibility  for  the  quality  of  the  data.  This  individual  leader  should  be  aware  of 
the  TQM  discipline  and  DAdm.  This  leader  will  help  top-management  establish  the  cultural 
environment  for  implementing  TDQM.  The  leader  establishes  a  formal  or  informal  DQE  team 
consisting  of  the  DAd,  DBAd  and  other  technical  and  functional  experts  and  has  the  overall 
responsibility  for  implementing  the  DoD  DQE  methodology.  Information  on  TDQM  efforts 
should  be  forwarded  by  the  DQE  team  leader  to  the  DoD  DAd  so  that  it  can  be  included  in 
annual  reports  and  included  in  future  strategic  plans. 


e.  The  DoD  DAd  will  provide  the  guidance  and  consultation  to  support  Steps  1 
and  2  in  a  DoD  TDQM  effort.  DoD  DAdm  establishes  the  overall  DoD  data  quality  assurance 
environment  and  will  assist  organizations  in  identifying  improvement  projects,  and  developing 
implementation  plans  and  prototype  efforts. 
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f.  As  part  of  the  review  process,  action  plans  should  be  reviewed  by  the  DoD 
DAd,  FDAds,  or  CD  Ads  for  the  achievement  of  data  quality  assurance  objectives.  These  action 
plans  should  have  met  the  data  quality  goal,  and  should  have  described  the  organization's 
resources,  tasks  and  milestones  needed  to  implement  the  data  quality  objectives  identified  in  the 
DASP  (reference  (c)).  Resource  requirements  should  be  analyzed,  in  reference  to  each  action 
plan,  to  determine  areas  of  improvement.  Training  resources  must  be  made  available  to  support 
the  organization’s  continued  TDQM  improvement  efforts.  Training  plans  should  be  updated  to 
provide  an  immediate  opportunity  to  use  new  skills  developed  throughout  the  TDQM  project. 


2.  Functional  and  Technical  Personnel  -  The  data  quality  assurance  leader  identified  in 
Step  1  of  the  TDQM  process  has  the  overall  responsibility  for  implementing  the  DoD  DQE 
methodology.  This  leader  establishes  a  formal  or  informal  DQE  team  consisting  of  the  DAd, 
DBAd  and  other  technical  and  functional  experts.  Throughout  the  DoD  DQE  effort,  the  leader 
meets  periodically  with  team  members  to  exchange  ideas,  review  interim  findings,  seek  answers 
to  specific  questions,  and  resolve  issues. 

a.  The  DQE  team  meets  regularly  throughout  the  DQE  process  and  are  considered 
a  dedicated  resource  until  the  process  is  complete.  The  Source  Data  Experts,  although  not  on 
the  DQE  team,  are  called  upon  throughout  the  DQE  process  for  consultation  and  validation 
during  the  DEFINE,  MEASURE  and  ANALYZE  activities,  and  to  implement  corrective  actions 
and  improvement  opportunities  during  the  IMPROVE  activity. 


b.  The  DQE  team  will  determine  the  specific  data  quality  problems  for  the 
assessment  and  gather  the  appropriate  data  experts  and  supporting  information.  The  DQE  team 
must  agree  on  a  clear  objective  for  the  data  quality  engineering  effort.  Once  the  objective  has 
been  determined,  the  DQE  team  should  identify  the  experts  (i.e.  end  users,  data  entry  clerks, 
DBAds,  programmers)  that  have  knowledge  of  the  how  the  data  values  are  created,  updated,  and 
maintained.  These  data  experts  should  also  understand  the  objective  and  be  available  for 
consultation  throughout  the  DQE  effort. 


c.  During  the  DQE  MEASURE  activity,  the  DQE  team  establishes  the  data  quality 
baseline  assessment.  Functional  and  user  feedback  is  vital  at  this  stage,  to  ensure  the  assessment 
data  is  valid  and  depicts  an  accurate  picture  of  the  real  data  quality  problems. 
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d.  As  DoD  legacy  systems  migrate  to  target  systems,  technical  developers  must 
build  data  quality  engineering  into  the  design  of  the  new  system,  thereby  phasing  out  the  need 
for  separate  DQE  tools.  For  non~automated  data  qualify  engineering  efforts,  the  DQE  team 
should  establish  standard  operating  procedures  with  users  for  detecting,  documenting,  and 
reporting  errors.  The  team  should  establish  the  measurement  technique  (i.e.  sampling)  and 
design  a  Data  Trouble  Report  (DTR)  along  with  instructions  for  use.  The  automated  DQE  tool 
should  be  configured  to  produce  graphic  and  tabular  data  error  reports. 


e.  As  part  of  the  DQE  ANALYZE  activity,  the  first  step  in  implementing  an 
effective  corrective  action  is  to  identify  the  data  source,  and  the  data  steward.  The  DoD  DQE 
team  examines  suspect  data  where  more  than  one  source  for  similar  data  exists  and  if  necessary, 
determines  the  best  source  for  the  data  based  on  the  quality  assessments  provided  by  the  DoD 
DQE  tool.  Data  stewards  or  experts  are  determined  to  identify  the  correct  people  required  to 
support  the  DQE  team  in  resolving  data  problems.  Individual  efforts  can  substantially  contribute 
to  the  data  quality  assurance  effort,  even  if  undertaken  outside  the  context  of  the  DQE  team. 
It  is  important  to  contact  and  involve  the  functional  data  stewards  and  functional,  technical,  and 
data  experts  who  work  with  the  data  or  are  in  the  process  every  day.  The  DQE  team  must 
depend  on  them  to  identify  causes  of  problems  with  their  data  and  solutions  or  opportunities  for 
improvements.  People  will  contribute  most  when  they  are  responsible  for  something. 
Therefore,  if  these  individuals  recognize  that  quality  is  their  responsibility  every  day,  it  will 
become  easy  to  accept  the  importance  of  being  involved  and  providing  resources  necessary  to 
improve  data  quality. 


f.  For  the  DQE  IMPROVE  activity,  the  DQE  team  must  formally  provide 
recommended  improvement  alternatives  to  one  or  all  of  the  following  (when  applicable):  data 
stewards.  Functional  Activity  Program  Managers  (FAPM),  Automated  Information  Systems 
Program  Managers  (AIS  PM),  and  users  depending  on  the  type  of  problem  (i.e.  system,  process, 
procedure  or  data  design).  Regardless  of  the  type  of  problem,  the  data  value  error  should  be 
formally  documented  and  provided  to  the  source  file  data  steward  or  DBAd  for  immediate 
corrective  action.  The  data  steward  or  DBAd  should  notify  the  DQE  team  when  the  error  is 
corrected.  If  a  solution  alternative  or  corrective  action  cannot  be  implemented,  then  an 
explanation  and  solution  alternatives  should  be  provided  to  the  DQE  team.  The  DQE  team 
should  document  all  implemented  solutions,  corrective  actions  taken,  and  explanations  in  the 
DQE  Data  Quality  Baseline  Assessment  Report. 
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g.  The  DQE  team  must  monitor  implementation  progress  of  all  improvement 
opportunities  and  corrective  actions  to  data  values.  An  improvement  opportunity  may  take 
longer  to  implement  than  the  time  allotted  for  the  DQE  project  due  to  the  time  needed  to  get 
plans  approved  and  to  allocate  resources.  In  this  case,  the  DQE  team  may  assign  individuals 
to  follow  up  on  the  improvements  periodically.  The  DQE  team  must  reassess  the  data  to 
determine  how  well  the  actual  performance  matches  the  planned  improvements.  Tracking  the 
effectiveness  of  improvement  efforts  is  actually  considered  a  part  of  the  measurement  and 
analysis  efforts.  After  the  improvement  and/or  corrective  actions  are  validated,  the  DQE  team 
should  document  any  improved  data  quality,  reduced  costs  and  the  DQE  performance  data 
thoroughly  in  the  Data  Quality  Baseline  Assessment  Report. 
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Producer  Data  Certification:  determination  by  the  data  producer 
data  have  been  verified  and  validated. 
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COMMUNICATION 

Data  Example:  Probability  of  communications  for  a  given  scenario. 
Process:  Determine  system/scenario  characteristics,  run  simulation. 
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AMSAA  AS  AN  AUTHORITATIVE  DATA  SOURCE 

DATA  VV&C  PROCESS 


DATA  VV&C  EXAMPLE  -  FARV  COEA 

•  TRAC-OAC  Receives  Task  to  Perform  COEA 
-  Prepares  Study  Plan 


-  840 


CO 


CO 


G 
O  c 

a  g 
g  a 

Vh  (D 

•3  S 
•  *— 1 

CT  G 

<d  cr 

g  w 

cS  5 
Q  G 

m  Q 
2  is 


■s  s 

§  -e 


*a  * 

o  h 


4— » 

Cw 

G 

cd 

G 

O 

+-> 

o 

OJD 

*-< 

G 

G 

•HH 

O 
•  ^ 

T3 

<+H 

rH 

■4— *  4— > 
t/3  1-H 

G 

O  <D 
P  ° 
C-  -q 

$  G 
G 

g  o 
+-»  o 

G  ut 

o  § 

co  co 

£  CD 
G  +-> 

5  g 

Q**-< 

8  &. 

cu  o 

<r>  a 

O  &■ 


a> 

©  < 
C/3 

232 

§-< 
2  ° 
G  -+-> 

— *  CO 

(3  0 
^  *0  G 
CT 
O 
u 


O 


<<2 
00  Sti 


I  4-rf  G 

U 

<  ® 

E-  1 


G 

4-» 

G 

p  « 


g  c 
Cl*  g 

£  p 
Ph  oo 


o 

<3 


o 

U 


o 

C/D 


T3 

O 

*o 


^G 

O 


o 

Ci) 

G 

y 

o 

G 

Ph 


C3  o 

Q  5 

C“ 

CO  CD 

O  u. 

i 

G  G 
Co  '*■* 

o  iS 

U 

Pi  CO 

<  I 

<•> 
oo  6 

2* 

<  ' 


C/D 

o  ^ 

s  a  *r 

3  g 

O 

CO  cd 

-U-3 
g>2£ 

•a  <  p 

3  o  (S 

i-  jz  +; 
o  — »  cd 

aO'c 
CO  G  T3 

2  5  t 

3<=  s 
—  — »  cr 

^  S  2 

G  Q*  *-* 

cd  S  co 

CO  G 

O  £  cd 
53  .3  ^ 

G  2  o 
>2  5  C 
i3  .o  p 

uoo 


-  Conducts  formal  data  review 

-  Prepares  and  signs  certification  letter 

-  Submits  data  package  electronically  in  TADS  format 
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APPENDIX  F:  ACRONYMS 


A&T 

Acquisition  and  Technology 

AAW 

Anti- Air  Warfare 

ACI 

A&T  CIM  Integration 

ADS 

Authoritative  Data  Sources 

ADT 

Abstract  Data  Type 

AF 

Air  Force 

AFC4A 

an  Air  Force  organization 

AI 

Artificial  Intelligence 

AIS 

Automated  Information  System 

AITS 

a  joint  program  office  that  supports  DMSO 

ALSP 

Aggregate  Level  Simulation  Protocol 

AMC 

Army  Material  Command 

AMIP 

Army  Model  Improvement  Program 

AMSAA 

Army  Material  Systems  Analysis  Activity 

AMSMO 

Army  Model  and  Simulation  Management  Office 

ANL 

Aberdeen  National  Laboratory 

ANSI 

American  National  Standards  Institute 

ARL 

ARL-IJT  Research  (name  of  a  company) 

ARMS 

Automated  Repository  for  Models  and  Simulations 

ARPA 

Advanced  Research  Project  Agency 

ASD 

Assistant  Secretary  of  Defense 

ASW 

Anti-Submarine  Warfare 

ATD 

Advanced  Technology  Demonstration 

ATO 

Afloat  Training  Organization 

BFTT 

Battle  Force  Tactical  Trainer  Project 

BINHEX 

Macintosh  format  for  sending  formatted  text 

BLOBS 

Binary  Large  Objects 

BMDO 

Ballistic  Missile  Defense  Organization 

C3I 

Command,  Control,  Communications  and  Intelligence 

CCTT 

Close  Combat  Tactical  Trainer  Project 

CDAd 

Component  Data  Administrator 

CDIF 

CASE  Data  Interchange  Format 

CENTCOM 

Central  Command 

-  842  - 


CEOS 

CFDB 

CGF 

Cl 

CIM 

CINC 

CISS 

CNA 

CNO 

COE 

COEA 

CONOPS 

COTS 

DA 

DAAC 

DAB 

DAPMO 

DASD 

DASP 

DAd 

DB 

DBAd 

DBMS 

DBTWG 

DDR&E 

DDRS 

DIA 

DIF 

DIS 

DISA 

DISO 

DISWG 

DMA 

DMSO 

DON 

DRB 


Committee  on  Earth  Observation  Satellites 
Conventional  Force  Data  Base 
Computer  Generated  Forces 
Catalog  Interoperability 

Corporate  Information  Management  and  Center  for  Information 
Management 

Commander-In-Chief 

Center  for  Information  System  Security 

Center  for  Naval  Analysis 

Center  for  Naval  Operations 

Common  Operating  Environment 

Cost  and  Effectiveness  Analysis 

Concept  of  Operations 

Commercial  Off  The  Shelf 

Data  Administration 

Distributed  Active  Archive  Centers 

Defense  Acquisition  Board 

Data  Administration  Program  Management  Office 

Deputy  Assistant  Secretary  of  Defense 

Data  Administration  Strategic  Plan 

Data  Administrator 

Data  Base 

Data  Base  Administrator 

Data  Base  Management  System 

Information/Data  Base  Technology  Working  Group 

Director  Development  Research  and  Engineering 

Defense  Data  Repository  System 

Defense  Intelligence  Agency 

Data  Interchange  Format 

Distributed  Interactive  Simulation 

Defense  Information  Systems  Agency 

Defense  Information  Services  Organization 

Distributed  Information  System  Working  Group 

Defense  Mapping  Agency 

Defense  Modeling  and  Simulation  Office 

Department  of  Navy 

Defense  Resources  Board 
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DSB 

DSCS 

DSI 

DTIC 

DoD 

DoDD 

E&E2 

E2DIS 

EA 

ECMA 

ECS 

EDHS 

EDI 

EOS 

EOSDIS 

EW 

EXCIMS 

FAPM 

FDAd 

FFRDC 

FIPS 

FORN 

FWG 

FY 

GCCS 

GPS 

HDF 

HQ 

HW 

IAC 

IC 

ICASE 

IDEF 

IDEFO 

IDEF  IX 

IDEF3 

IDEF4 


Defense  Science  Board 
Data  Centers  Standards  Committee 
Defense  Simulation  Internet 
Defense  Technical  Information  Center 
Department  of  Defense 
Department  of  Defense  Directive 
Environment  and  Environmental  Effects 

Environmental  Effects  in  Distributed  Interactive  Simulation  Project 
Executive  Assistant 

European  Computers  Manufacturers  Association 

EOSDIS  Core  System 

ECS  Data  Handling  System 

Electronic  Data  Interchange 

Earth  Observation  Satellite 

Earth  Observation  Satellite  Distributed  Information  System 
Electronic  Warfare 

Executive  Council  for  Modeling  and  Simulation 

Functional  Activity  Program  Manager 

Functional  Data  Administrator 

Federally  Funded  Research  and  Development  Center 

Federal  Information  Processing  Standard 

Foreign 

Functional  Working  Group 
Fiscal  Year 

Global  Command  and  Control  System 
Global  Positioning  System 
Hierarchical  Data  Format 
Headquarters 
Hardware 

Information  Analysis  Center 
Intelligence  Community 

Integrated  Computer  Aided  Software  Engineering 
Integrated  Computer  Aided  Definition  Language 
IDEF  process  modeling  methodology 
IDEF  data  modeling  methodology 
IDEF  process  modeling  methodology 
IDEF  object  oriented  modeling  methodology 
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IDEFxx 

IDEF  whatever 

IDL 

Interface  Definition  Language 

IDN 

International  Directory  Network 

IEC 

International  Electrotechnical  Commission 

IM 

Information  Manager 

INFORMIX 

name  of  a  commercial  relational  database  management  system 

INFOSPAN 

name  of  a  commercial  information  resource  dictionary 
system  product 

IPL 

Integrated  Project  List 

I  FI’s 

Integrated  Process  Team 

IRDS 

Information  Resource  Dictionary  System 

IRR 

Information  Resource  Repository 

ISO 

International  Organization  of  Standardization 

ITSEC 

Interservice/industry  training  systems  and  education  conference 

JCS 

Joint  Chief  of  Staff 

JDBE 

Joint  Data  Base  Elements  Project 

JESEBEL 

Joint  Electromagnetic  Signature  and  Effects  Database  Library 

JIEO 

Joint  Interoperability  Engineering  Organization 

JMCIS 

Joint  Marine  Corps  Information  System 

JOPES 

Joint  Operation  Planning  and  Execution  System 

JPL 

Jet  Propulsion  Laboratory 

JPO 

Joint  Project  Office 

JROC 

Joint  Requirements  Oversight  Council 

JS 

Joint  Staff 

JSIMS 

Joint  Simulation  System  Project 

JTAMS 

Joint  Tactical  Missile  Signatures  Program 

JTC1 

Joint  Technical  Committee  l 

JUDI 

Joint  Universal  Data  Interpreter 

KBSI 

name  of  a  company 

LLNL 

Lawrence  Livermore  National  Laboratory 

M&S 

Modeling  and  Simulation 

MAISRC 

Major  Automated  Information  System  Review  Council 

MAJCOM 

Major  Command 

MC&G 

Mapping,  Charting  and  Geodesy 

MCEB 

Military  Communications  and  Electronics  Board 

MEL 

Master  Environmental  Library  Project 

MLS 

Multi-Level  Security 

MOA 

Memorandum  of  Agreement 

MORS 

Military  Operations  Research  Society 

MS&A 

Modeling,  Simulation  and  Analysis 

MSWG 

Modeling  and  Simulation  Working  Group 

NASA 

National  Aeronautics  and  Space  Administration 

NAVEUR 

Navy  Europe 

NDI 

Non  Developmental  Item 

NIST 

National  Institute  of  Standards  and  Technology 

NO  A  A 

National  Oceanographic  and  Atmospheric  Agency 

NRL 

Naval  Research  Laboratory 

NRaD 

Navy  Research  and  Development 

NSI 

NASA  Science  Internet 

NSS 

National  Security  Strategy 

NWTDB 

Navy  Warfare  Tactical  Database 

OASIS 

Operational  Analysis  and  Simulation  Interface  System 

ODBC 

Object  Data  Base  Connectivity  standard 

ODDRE 

Office  of  the  Director  Development,  Research  and  Engineering 

ODISC4 

Office  of  the  Director  of  Information  Systems  Command, 
Control,  Communications  and  Computers 

OMC 

Object  Management  Committee 

OMG 

Object  Management  Group 

ONR 

Office  of  Naval  Research 

OO 

Object  Oriented 

OODBMS 

Object  Oriented  Data  Base  Management  System 

OPLAN 

Operational  Plan 

ORG 

Organization 

OSD 

Office  of  the  Secretary  of  Defense 

OUSD 

Office  of  the  Under  Secretary  of  Defense 

P&L 

Production  and  Logistics 

PCTE 

Portable  Common  Tool  Environment 

PDU 

Protocol  Data  Unit 

PEO 

Program  Executive  Office 

PM 

Program  Manager 

POC 

Point  of  Contact 

PPBS 

Planning,  Programming  and  Budgeting  System 

PSA 

Principal  Staff  Assistant 

PSAG 

Phenomenology  Science  and  Analysis  Group 
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Ph 

Probability  Hit 

Pk 

Probability  Kill 

RCL 

Rule  Constraint  Language 

RDBMS 

Relational  Data  Base  Management  System 

RFC 

Requests  for  Changes 

RFP 

Request  for  Proposal 

S&T 

Science  and  Technology 

SAFOR 

Semi  Automated  Forces 

SAG 

Senior  Advisory  Group 

SAI 

Subject  Area  Information 

SASC 

Senate  Armed  Services  Committee 

SBIS 

Sustaining  Base  Information  System 

SC 

Standing  Committee 

SDS 

Software  Distribution  System 

SGI 

Stilicon  Graphics  Inc. 

SGML 

Standard  Genealized  Markup  Language 

SIG 

Special  Interest  Group 

SIM 

Simulation 

SIMDAT 

Simulation  Data 

SIMDATAM 

Simulation  Data  Management 

SOCOM 

Special  Operations  Command 

SQL 

Standard  Query  Language 

SROC 

Senior  Readiness  Oversight  Council 

STRICOM 

Simulation  Training  and  Instrumentation  Command 

T&E 

Test  and  Evaluation 

TAP 

Technology  Area  Plan 

TAFIM 

Technical  Architecture  Framework  for  Information  Management 

TBD 

To  Be  Decided 

TC 

Technical  Committee 

TF 

Task  Force 

TGOO 

Technical  Group  for  Object  Oriented 

TMS 

Tactical  Missile  Signature 

TRAC 

Training  Command 

TRADOC 

Training  and  Doctrine  Command 

TTS 

Tactical  Training  Strategy 

TWG 

Technical  Working  Group 

UPIG 

User  Products  Information  Group 
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URL 

Universal  Resource  Locator 

USACOM 

US  Atlantic  Command 

USAFE 

US  Air  Force  Europe 

USAREUR 

US  Army  Europe 

USD 

Under  Secretary  of  Defense 

USMC 

US  Marine  Corp 

UTM 

Universal  Transverse  Mercator 

UTSS 

Universal  Threat  System  for  Simulators  Project 

V&V 

Verification  and  Validation 

VV&A 

Verification,  Validation  and  Accreditation 

vv&c 

Verification,  Validation  and  Certification 

WAIS 

Wide  Area  Information  Server 

WG 

Working  Group 

WPC 

Warrior  Preparation  Center 

WWMCCS 

World  Wide  Military  Command  and  Control  System 

WWW 

World  Wide  Web 

X3H2 

an  ANSI  standards  group 

X3H4 

an  ANSI  standards  group 

X3L8 

an  ANSI  standards  group 

XOM 

Air  Force  modeling  and  simulation  office 

XPSD 

Air  Force  organization 

